Virtual information board for collaborative information sharing

ABSTRACT

An information board is displayed and used in a three-dimensional virtual reality environment wherein the information board is extended by adding discrete board segments through control inputs received from a user in the virtual reality environment.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims the benefit of priority of U.S.Provisional Application No. 63/121,828, filed Dec. 4, 2020, which isincorporated herein by reference in its entirety.

BACKGROUND OF THE INVENTION

Virtual reality (VR) environments have grown more realistic andimmersive as VR headsets, processor speeds, data storage and datatransfer technologies have continued to improve. However, unlikeconventional physical reality, electronic VR environments present moreopportunities for persons to collaborate and share information,including in work and education fields, in ways that are not possible inthe physical constraints of the real-world.

One of the challenges to taking advantage of the these opportunities iscombining two-dimensional (2D) and three-dimensional (3D) displays ofvirtual objects and information to enable users to see, use, store andmanage collaborative activities, information, other users and thevirtual objects optimally in the VR environment. Another challenge toworking and learning in VR environments is adapting a user's real-worldphysical positions and motions to provide information inputs andcorresponding VR experiences and VR views in the virtual reality world.

SUMMARY OF THE INVENTION

Embodiments of the invention address these challenges by providingmethods and systems with improved display and functionality for usinginformation boards in a VR environment. In embodiments, methods andsystems of the invention are implemented through development tools forthe Oculus/Meta Quest platform (Oculus Platform SDK) by Oculus VR(Irvine, Calif.) (parent company Meta). It will be appreciated that thesystems and methods, including related displays, user interfaces,controls and functionalities, disclosed herein may be similarlyimplemented on other VR platforms with other VR SDKs and softwaredevelopment tools known to VR developers.

In one embodiment, a method for providing an information board in avirtual reality environment comprises displaying an information boardincluding a first discrete segment viewable in a three-dimensionalvirtual reality environment; receiving a control input adding a seconddiscrete segment to the information board; and displaying the seconddiscrete segment adjacent to the first discrete segment in the virtualreality environment in response to receiving the control input.

In a further embodiment, a method of the invention further includesreceiving first viewable information on the first discrete segment andsecond viewable information on the second discrete segment and storingin a database accessible by the virtual reality environment sequentialordering information for the information board that the first viewableinformation is linked to be displayed on the first discrete segment, thesecond viewable information is linked to the be displayed on the seconddiscrete segment, and the second discrete segment was created after thefirst discrete segment.

In a further embodiment, the first discrete segment has a left side andright side from a user's perspective in the virtual reality environmentand the second discrete segment is displayed after and to the right ofthe first discrete segment following receipt of the control input addingthe second discrete segment.

In another embodiment, an inventive method further comprises receivingadditional respective control inputs to add respective multiple discretesegments of the information board and displaying respective multiplediscrete segments of the information board in the virtual realityenvironment in a sequential order following first and second discretesegments after the respective additional control inputs are received.

One embodiment of a method of the invention further comprises displayingthe information board in a room with walls in the virtual realityenvironment and adjusting the size and display of the room whenrespective multiple discrete segments are displayed.

Another embodiment of a method of the invention includes displaying ananimation of one or more discrete segments of the information boardcollapsing into the information board.

In embodiment of the invention, an information board may be a virtualrepresentation of a variety of known physical boards or displays, and asegment of the information board may be displayed in the virtualenvironment as one of a whiteboard, chalkboard, grid with gridlines, andgrid with dotted markers following receipt of control input selecting aninformation board segment type for a segment from a displayed graphicaluser interface.

In embodiments of the invention, a method for providing an informationboard in a virtual reality environment comprises displaying a graphicaluser interface for a user to select a first control input to add adiscrete segment to an information board in a three-dimensional virtualreality environment; displaying a graphical user interface for a user toselect a second control input corresponding to a type of informationboard segment when adding the discrete segment to an information boardin a three-dimensional virtual reality environment; and receivingrespective multiple first control inputs to add respective multiplediscrete segments of the information board, receiving respectivemultiple second control inputs selecting a type of information boardsegment for each added respective discrete segment, and displayingrespective multiple discrete segments of the information board in thevirtual reality environment in a sequential order.

In a further embodiment of the invention, a method for providing aninformation board in a virtual reality environment further comprisesstoring in a database the sequential order of the display of themultiple discrete segments of the information board; and followingreceipt of a third control input requesting retrieval of the informationboard after a user has left and returned to the virtual realityenvironment, displaying a plurality of the multiple discrete segments insequential order in the virtual reality environment.

In another further embodiment, a method of the invention furthercomprises displaying respective viewable information content associatedwith each respective discrete segment of the plurality of the multiplediscrete segments displayed in sequential order.

In other embodiments, a method of the invention includes re-sizing of aroom in which the information board is displayed in the virtual realityenvironment in connection with displaying the respective multiplediscrete segments of the information board.

A system for displaying an information board in a three-dimensionalvirtual reality environment is provided in one embodiment that comprisesa graphical user interface displayed in the virtual environmentconfigured to receive control inputs to add display of discrete segmentsof the information board in the virtual reality environment in asequential order; and a virtual room that is viewable together with thegraphical user interface in the virtual reality environment, wherein thevirtual room is configured to re-size when a number of discrete segmentsof the information board have been added to the information board thatwould result in the information board appearing to extend beyond thewalls of the virtual room.

In a further embodiment, a system of the invention further comprises aninformation board that includes multiple discrete segments sequentiallyordered from left to right based on the time of addition to theinformation board from the perspective of a user in the virtual room.

In a further embodiment, a system of the invention further comprises adatabase operatively linked to the information board that storesrespective persistent two-dimensional information corresponding to eachrespective discrete segment of the information board.

In a further embodiment, a system of the invention further comprises aphysical controller configured for the user to add the persistenttwo-dimensional information to corresponding discrete segments of theinformation board in the virtual reality environment.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be more clearly understood from the followingdescription given by way of example, in which:

FIG. 1 is a schematic diagram illustrating a user with an informationboard in virtual reality environment in an embodiment of the invention.

FIG. 2 is a schematic diagram illustrating an information board withmultiple adjacent discrete segments in a virtual reality environment inan embodiment of the invention.

FIG. 3 is a schematic diagram illustrating an in-application informationboard tool palette in a virtual reality software application in anembodiment of the invention.

FIG. 4 is a schematic diagram illustrating a user interacting with abutton in the in-application information board tool palette of FIG. 3 inan embodiment of the invention.

FIG. 5 is a schematic diagram illustrating a user pressing athree-dimensional button in a virtual reality environment that activatesa board segment to slide out of an information board in the virtualreality environment in an embodiment of the invention.

FIG. 6 is a schematic diagram illustrating an information board withmultiple segments in which information is added and/or displayedwest-to-east in a virtual reality environment platform and re-orientedto information board segments in which information is added and/ordisplayed north-to-south in a another platform, such as GoBoard.com. inan embodiment of the invention.

FIG. 7 is a schematic diagram illustrating cropping of informationacross segments of the west-to-east information board for display on thenorth-to-south information board segments of FIG. 6 in an embodiment ofthe invention.

FIG. 8 is a schematic diagram illustrating an information board of auser's physics' room having a library background environment setting ina virtual reality environment in an embodiment of the invention.

FIG. 9 is a schematic diagram illustrating an in-application toolpalette for selecting a background environment setting in a virtualreality environment in an embodiment of the invention.

FIG. 10 is a schematic diagram illustrating side-by-side examples of aone-sided, two-sided and 5-sided in virtual information board in avirtual reality environment in an embodiment of the invention.

FIG. 11 is a schematic diagram illustrating three-dimensional displayedinformation converting to two-dimensional displayed information forstorage on an information board in a virtual reality environment in anembodiment of the invention.

FIG. 12 is a schematic diagram illustrating a user pulling aninformation board around the user in a virtual reality environment in anembodiment of the invention.

FIG. 13 is a schematic diagram illustrating a user pulling aperpendicular segment from an information board in a virtual realityenvironment in an embodiment of the invention.

FIG. 14 is a schematic diagram illustrating a user viewing multipleinformation board segments from an information board above the user'shead in a virtual reality environment and the segment that ishighlighted or marked is displayed in front of the user in an embodimentof the invention.

FIG. 15 is a schematic diagram illustrating a user turningcounterclockwise and moving or rotating multiple information boardsegments from an information board above the user's head in a virtualreality environment and the segment that is highlighted or marked issubsequently displayed in front of the user in an embodiment of theinvention.

FIG. 16 is a schematic diagram illustrating a user selecting a segmentfrom multiple information board segments from an information board abovethe user's head in a virtual reality environment and the segment that ishighlighted or marked is subsequently displayed in front of the userwith adjacent board segments in an embodiment of the invention.

FIG. 17 is a schematic diagram illustrating multiple types ofinformation board segments to be primary items in an information boardthat displayed as user looks downward, such as at the user's feet, in avirtual reality environment and an information board segment willinclude the type of primary item selected by the user in an embodimentof the invention.

FIG. 18 is a schematic diagram illustrating a user turning to face atype of information board segment to be a primary item in an informationboard in a virtual reality environment in an embodiment of theinvention.

FIG. 19 is a schematic diagram illustrating a user's selection of aprimary item for an information board segment in a virtual realityenvironment and an information board segment and the type of primaryitem selected by the user is inserted in the information board while theprior primary item and segment is move out of sight of the user in anembodiment of the invention.

FIG. 20 is a schematic diagram illustrating a user using informationboard segments of an information board as walls in a virtual realityenvironment that correspond to physical walls of the user's physicalreality in an embodiment of the invention.

FIG. 21 is a schematic diagram illustrating users collaborating to shareinformation on an information board in respective virtual realityenvironments wherein each user has a different background environmentsettings in their respective VR environments while the information onthe board remains the same for each user in an embodiment of theinvention.

FIG. 22 is a schematic diagram illustrating the transition of a segmentof an information board in a virtual reality environment betweenhorizontal and vertical orientations in an embodiment of the invention.

FIG. 23A is a schematic diagram illustrating an information board in avirtual reality environment in which a virtual desk appears to extendthe same length as the information board in an embodiment of theinvention.

FIG. 23B is a schematic diagram illustrating an information board in avirtual reality environment in which a virtual desk appears to be a setsize while the information board appears to extend as long as theinformation board holds in an embodiment of the invention.

FIG. 24 is a schematic diagram illustrating an information board in avirtual reality environment used at a 45 degree angle in an embodimentof the invention.

FIG. 25 is a schematic diagram illustrating automatic adjustment ofdisplay of an information board based on different heights of users inan embodiment of the invention.

FIG. 26 is a schematic diagram illustrating automatic adjustment ofdisplay of an information board and users to each respective users'perspective in a virtual reality environment based on differentreal-world body positions of users in an embodiment of the invention.

FIG. 27 is a schematic diagram illustrating a user seeing a first-personrealistic view of self together with an information board in a virtualreality environment in an embodiment of the invention.

FIG. 28 is a schematic diagram illustrating a user otherwise blockinganother user's view of an information board in a virtual realityenvironment being displayed as transparent in an embodiment of theinvention.

FIG. 29 is a schematic diagram illustrating a user watching atwo-dimensional video replay of that user's interaction or collaborationwith an information board in a virtual reality environment from a thirdparson and two-dimensional perspective in an embodiment of theinvention.

FIG. 30 is a schematic diagram illustrating a user watching an immersiveand three-dimensional replay of that user's interaction or collaborationwith an information board in a virtual reality environment from a thirdparson and three-dimensional perspective in an embodiment of theinvention.

FIG. 31 is a schematic diagram illustrating a display to a user of aninformation board in a virtual reality environment of a meter showingengagement of other users of the information board in an embodiment ofthe invention.

FIG. 32 is a schematic diagram illustrating a display to a user of aninformation board in a virtual reality environment of a meter showingcognitive load of other users of the information board in an embodimentof the invention.

FIG. 33 is a schematic diagram illustrating a display to a user of aninformation board in a virtual reality environment of a meter showingengagement and cognitive load of different respective users of theinformation board in an embodiment of the invention.

FIG. 34 is a schematic diagram illustrating a user of an informationboard in a virtual reality environment with a device connecting to a VRheadset that provides writing and drawing input to the information boardin an embodiment of the invention.

FIG. 34 is a perspective view of a wireless pen mouse device connectingto a VR headset that provides writing and drawing input to theinformation board in an embodiment of the invention.

FIG. 36 is a schematic diagram illustrating a user of an informationboard in a virtual reality using a triple functionality gesture of laserpointer, viewing screen, and teleporting in an embodiment of theinvention.

FIG. 37 is a schematic diagram of a remote information sharing systemfor users in a shared virtual reality environment in which each useruses a respective locally available computer running virtual realityapplication software to share viewable information in the virtualreality environment in an embodiment of the invention.

FIG. 38 is a schematic diagram of a remote information sharing systemfor users in a shared virtual reality environment in which each useruses a remote server computer running virtual reality applicationsoftware to share viewable information in the virtual realityenvironment in an embodiment of the invention.

DETAILED DESCRIPTION

For clarity of explanation, in some instances, the present technologymay be presented as including individual functional blocks includingfunctional blocks comprising devices, device components, steps orroutines in a method embodied in software, or combinations of hardwareand software.

Any of the steps, operations, functions, or processes described hereinmay be performed or implemented by a combination of hardware andsoftware services or services, alone or in combination with otherdevices. In some embodiments, a service can be software that resides inmemory of a client device and/or one or more servers of a contentmanagement system and perform one or more functions when a processorexecutes the software associated with the service. In some embodiments,a service is a program or a collection of programs that carry out aspecific function. In some embodiments, a service can be considered aserver. The memory can be a non-transitory computer-readable medium.

In some embodiments, the computer-readable storage devices, mediums, andmemories can include a cable or wireless signal containing a bit streamand the like. However, when mentioned, non-transitory computer-readablestorage media expressly exclude media such as energy, carrier signals,electromagnetic waves, and signals per se.

Methods according to the above-described examples can be implementedusing computer-executable instructions that are stored or otherwiseavailable from computer-readable media. Such instructions can comprise,for example, instructions and data which cause or otherwise configure ageneral purpose computer, special purpose computer, or special purposeprocessing device to perform a certain function or group of functions.Portions of computer resources used can be accessible over a network.The executable computer instructions may be, for example, binaries,intermediate format instructions such as assembly language, firmware, orsource code. Examples of computer-readable media that may be used tostore instructions, information used, and/or information created duringmethods according to described examples include magnetic or opticaldisks, solid-state memory devices, flash memory, USB devices providedwith non-volatile memory, networked storage devices, and so on.

Devices implementing methods according to these disclosures can comprisehardware, firmware and/or software, and can take any of a variety ofform factors. Typical examples of such form factors include servers,laptops, smartphones, small form factor personal computers, personaldigital assistants, and so on. The functionality described herein alsocan be embodied in peripherals or add-in cards. Such functionality canalso be implemented on a circuit board among different chips ordifferent processes executing in a single device, by way of furtherexample.

The instructions, media for conveying such instructions, computingresources for executing them, and other structures for supporting suchcomputing resources are means for providing the functions described inthese disclosures.

In various embodiments, methods and systems of the invention arepreferably implemented through development tools for the Oculus/MetaQuest platform (Oculus Platform SDK) by Oculus VR (Irvine, Calif.)(parent company Meta). It will be appreciated that the systems andmethods, including related displays, user interfaces, controls andfunctionalities, disclosed herein may be similarly implemented on otherVR platforms with other VR SDKs and software development tools known toVR developers.

Virtual Reality (VR) Board & Classroom Ecosystem

Referring to FIG. 1, in a programmed virtual reality softwareapplication in an embodiment of the invention, a virtual realityenvironment is a customizable room, such as a classroom, conferenceroom, office and the like, with an information board 10 (whiteboard,chalkboard, smart board or a blackboard and the like) at the front ofthe room. When a new room is created and a user 5 is in the virtualreality environment, the whiteboard is a minimum size of 30 ft of boardspace across the front of the room, and this whiteboard is the focalpoint in the room. This 30 ft is the equivalent of five 6 ft discreteboard segments 15, as shown in FIG. 2, where each discrete board segment15 “joins” or is linked to the segments adjacent either side of suchsegment, to create one, continuous whiteboard 10. In this example thereare five (5) discrete board segments 5 of the continuous informationboard 10, and various functionalities as to the board segments aresubsequently disclosed, such as laser pointer triple functionality morefully explained herein.

The room itself is longer on each end than the whiteboard, with an extra6 ft either side for example, but depending on the background, this roomcould feel significantly larger, for example if a user has chosen to bein space or underwater, then this background appears to extendinfinitely into the distance on either side of (and behind) thewhiteboard. This virtual whiteboard can be customized in many ways, forexample to appear as a whiteboard, a chalkboard, a grey board with gridlines or dotted grid markers, among others. Each user can choose to viewthe virtual board and virtual room differently, but the content on theboard will stay uniform for each user (if one user draws with a bluemarker on a whiteboard, this drawing will appear in blue for any otheruser regardless of the kind of board they have selected to look at—itwill appear as blue chalk on a chalkboard, or blue marker on awhiteboard). This uniformity of content is important for the purpose oftutoring, because a tutor must be able to refer to the content of thewhiteboard by its color such as: “fill in the blanks in the blue text”,and each person in the room can identify the blue text easily.

Preferably, an information board in a virtual reality environment inembodiments of the invention maintains persistent content, orderedchronologically over time. On the board itself, the content is stored in2D, which creates a sense of linear chronology (this is rare in VR,where space is infinite in all directions). As a user comes back intothe room/board at later times (such as perhaps week-on-week), all theinformation on the board persists, and the user can continue addingcontent to the board, and adding new board segments, and the contentwill persist (stored, e.g., in persistent storage of a remote computerserver on a wide area network, but might also be locally stored incomputer storage media). Since the content persists on the informationboard, users can continue working with it over time, either alone, orwith others, such as a teacher, a tutor, peers, colleagues, or any otherpersons using or collaborating with the information board in the samevirtual reality environment.

In some embodiments, hashtags can be used to index informationassociated with the board, and act an easy way for users to search forprevious work that they have done or information that has been presentedto the user. These hashtags can be accessed in certain embodiments in auser palette utilities page. This page will bring up a list of all thehashtags used in the virtual room, and when selected, the customizationsection of the palate will show all instances of that hashtag being usedin the room. Even without hashtags, in alternative embodiments a usermay execute a search on the board content through the search bar in theutilities section on their user palette, which would search the boardfor all instances of the searched word.

Extending Boards and Re-Sizing Rooms

In embodiments, an information board, e.g. whiteboard/blackboard, can beextended by pressing a button 35 located on the users' in-app toolpalette 30 as shown in FIG. 3. In embodiments, buttons may additionallybe provided or in the alternative be provided at the ends of the boardas shown in FIG. 5, such as a virtual button 50 that mimics a physicalbutton and hovers in space where it can be pressed to extend a boardsegment. It will be appreciated, that numerous alternatives to buttonsas control inputs to activate the “added board segment” function mayalso be implemented, such as pulling levers, sliding mechanisms, and thelike.

An example button 35 in the user palette 30 for added board segments maybe as shown in FIG. 4. The palette itself is similar to a tablet, buthovers in the user's virtual room in the virtual environment. In thisinterface, the user can see a preview of their boards in the palette,and can select where on the board they would like to add a new boardsegment; whether that is at either end of the total board, or somewherein between two existing board segments.

The user simply clicks a button 35 on their palette (FIG. 4), or the VRbutton 50 at the edge of the board if the user is standing next to it(FIG. 5) (such as if the user has used up existing board space), and theboard extends itself, another six feet board segment 15, another sixfeet board segment 15, another six feet board segment 15, another sixfeet board segment 15, etc., for each press of the button. Inembodiments, as the information board is extending, the whole classroom(or other VR environment walls defining the space where the board islocated) is also extending in identical, proportionate increments (i.e.to maintain the ratio of the board taking up the majority of the room);up to one, two, ten miles long, and theoretically even infinitely.

The button at either end of the board looks and acts like a button inthe physical world; but in VR, may be mapped on X-Y-Z coordinates. FIG.5 shows a user pressing the button 50 in VR with hand tracking, as ifthey were pushing a button in the real-world. As the user presses thebutton, it depresses just like a button in the real-world would. Whenthe user lifts their finger off the button, the board segment 15 beginsto slide out, as seen in the second image. This board segment 15 slidesout 6 ft and then slots into place in line with the board segment it hasextended from as part of the continuous information board 10 (FIG. 2).The button 50 remains at the end of the newly added board segment sothat the user can continue to extend the board if they wish.

In another embodiment, the room doesn't increase in size every time theboard increases in size. Instead, the room increases in size only whenthe board would otherwise go through the walls of the room. At thatpoint where the board would reach the walls of the VR environment, theroom then extends, preferably at the same rate and ratio as the board.

Deleting Boards and Room Re-Sizing

Users can also choose to delete board sections. As a user deletessegments of the board, the room may be programmed to shrink, such as tomaintain the same proportions of information board to room walls aspreviously described. In other words, the board remains the main focalpoint of the room and the room is slightly larger than the board, andthat proportional display in the VR environment is maintained whetherthe board is expanded or reduced in size.

In embodiments, it can be important that a VR room is programmed toshrink as the boards are deleted. For example there might be fivelectures leading up to an exam, and the teacher indicates one of thelectures will not be covered by the upcoming exam; then one of the fiveboards could be deleted and the 30 ft board turns into a 24 ft board,with the room shrinking accordingly. Otherwise the user will end up witha room that is proportioned for a 30 ft board, but the board is now muchshorter than 24 ft, and there would be excess space at the edges of theboard. Essentially the concept here is that the board is atwo-dimensional representative of the three-dimensional space; the boardmaintains chronology of work over time and the room expands and retractsin accordance with the size of the board.

The user interface and prior explanation of controls for deleting aboard segment is similar functionally with using a control input, suchas button pressing, pulling a lever, sliding a control and the like, asfor extending a board segment.

Persistence of Content

In an illustrative scenario, every week a student and a tutor may meettogether in VR and keep adding more and more board space and keepworking linearly down the board as they add more content. For example, auser may be learning math, and each week the user meets with the user'stutor to go over more math content. In the first week the user mightstudy Algebra chapter 1, so the user will import the theory from chapter1 of the user's textbook onto the board, and then the user will extendthe board to work on definitions, extend the board again to write outpractice problems, and extend the board once more to write a summary ofthe user's chapter 1 lesson. Then, the following week when the user meetwith the user's tutor again, the user will extend the board by howevermany segments the user need to continue working with chapter 2, but theuser's work on chapter 1 persists; that is, it stays in the VR room andon the board, waiting for the user to go back to it. Every unique roomof a student and tutor is automatically saved so that all informationpersists. As the user are continually making the board bigger, the roomis getting bigger automatically too, on the same axis as the board,horizontally. The room continues to stretch without lookingdisproportionately stretched.

Connection to GoBoard (Study Edge, LLC, Gainesville, Fla.)

In embodiments, an information board in a VR embodiment may be connectedto an external, existing, website-based platform providing 2-dimensionalinteractive canvases called GoBoard.com (see www.goboard.com). A GoBoardis a vertically scrolling drawing canvas for people to collaborate foronline tutoring. While the VR information boards of the inventionpreferably extend from West to East, GoBoard extends North to South.Accordingly, in embodiment of the invention the content displayed bothon a GoBoard and in a VR information board of embodiments of theinvention is re-oriented as necessary to facilitate integration betweenthe VR environment and the GoBoard platform. A user can't just turn anentirety of a VR document file from West to East to North to South allat once, because if the user just rotates the file from one position,all of the pages would be disoriented by 90 degrees as the user scrolldown on GoBoard.

Referring to FIG. 6, each board segment 15 of the information board 10in VR is programmed to rotate individually, so that it works mosteffectively for display content with a GoBoard whiteboard 60 and GoBoardboard segments 65. This style of rotation would also be compatible witha Google sheet or Google document, in other embodiments. In stillfurther embodiments, it will be appreciated that similar re-orientationof West to East content for display in North to South contentenvironments can be similarly implemented between a VR environment withan information board and other information sharing applications andplatforms. In embodiments, each individual VR board segment isreoriented from horizontal to vertical. So when the information boardcontent from VR is to displayed in GoBoard, or Google docs, or printingto a physical printer, each of the VR board segments, which arehorizontal (East to West) are now vertical (North to South). Each of the6 ft boards which are oriented East to West in the VR room, are alsosegmented by the board lines at each 6 ft segment. So when the VR boardsegment content is to be displayed to GoBoard, each individual VR boardsegment will rotate its orientation as per the boardlines, and then berealigned in the North to South orientation for GoBoard web browserformatting. When the content of the board segments are displayed backinto a VR environment, such as after editing the content in the GoBoard2D application, the VR board segments will reorient back to West to Eastcontent display according to the board segmentation again. Accordingly,continuity of the content on each of the board segments is preservedwhen the content is exchanged from VR to GoBoard and back again.

Another embodiment by which the content is re-oriented from West-to-East(VR) to North-to-South (GoBoard), is to crop the VR board segments tothe nearest possible VR board segment, without allowing a VR boardsegment to disrupt content which extends over more than one VR boardsegment. Referring to FIG. 7, instead of being re-oriented at the end ofBoard A, the board content is instead cropped at the location of thedotted line 70 (illustrated to show location example), and then shrunkdown (the content is smaller in the North-to-South re-orientation) tofit on Board A segment in the North-to-South orientation of GoBoard. Theboard size is detected, and the content is reduced by the rationecessary to fit the VR board size. The ratio can be determined eitherby the width or height limitation. For purposes of illustration, this issimilar to taking a screenshot or image, and resizing all aspects of theimage so that the image retains proportion. The break between VR boardsmay also be based on the detection of whitespace (e.g. a departure fromcontent that is based on image and text recognition.)

Users Making Multiple Rooms

In embodiments, the VR environment is programmed so that users can makemultiple rooms (each with a VR information board) and there are severaldifferent conventions for the creation and persistence of content ofthese rooms. Information on the board preferably should persist, butalso information relating to the selected background, brightness,spatial audio, the avatars of each user, their selected tools, resourcesin use; all need to change between rooms and persist depending on who isin the room. This information does not need to be saved by the userbetween each session, but instead it automatically persists in the roomin which the user made the changes. The purpose for this functionalityis that if User A spends time setting up “User A's Physics Room”, forexample, User A wants those settings to remain when he or she goes backinto that room later. Similarly, he or she wants to be able to create adifferent room, for example “User A's History Room” and have differentsettings to the Physics Room. User A may want to be at the beach in theHistory Room, but feels more focused when in a library as experienced byimplementing background 80 with library display elements as part of thePhysics Room (as shown in FIG. 8). User A's room settings can be changedin his or her user palette 30 (as shown in FIG. 9), which is also whereall User A's tools and resources are kept.

Further embodiments of this concept are as follows:

i. The rooms may all have a unique identifier (e.g., a 7-digitalphanumeric code) to identify them. The user can then assign a vanityname according to the subject studied in that room: for example “UserA's Physics Room” “User A's Chemistry Room”. The people given access tothat room are people associated with each subject: e.g. User A, his orher tutor, maybe his or her teacher in each subject area. Whenever usersre-enter that 7-digit coded room (or the associated vanity name) fromtheir “recent rooms” section in the VR application, all information fromthat room persists accordingly.

ii. Another embodiment is that the room settings are created based onthe people in the room; not the room number. For example, if User A andUser B have met previously, and changed their room settings, then whenthe same two people, User A and User B, meet again, the same settingsthat they applied last time they met together would automatically beenabled. This applies to the tools used, their avatar appearance, theboard content, and the room settings, among other customizable toggleon/off features. This also means that all of their previously sharedfiles are available, their chat history, and everything else that haspreviously been shared between those two users.

iii. Different embodiment for when the users have the same tutor-studentcombo across two different subjects, for example User A is User B's Mathtutor, but also her Chemistry tutor. In this case there will bedifferent setting configurations for the same people combos, and whenthose two users join, they can switch easily between their tworooms/boards. The users are able to automatically retrieve a previoussession (including its settings, files, controls) when the same peopleare in the room. The user can also retrieve manually, for example ifUser A and User B start up a session, the VR application in anembodiment of the invention identifies the users, and it searchesthrough all the sessions that User A and User B had both been in, toprovide them with a selection of sessions by name or date. User A andUser B can then pick a session, or a whiteboard that they want to talkto in that session. They can bring up that old session, along with allthe settings and files associated with it when it was previouslyestablished.

iv. When an additional user is added to the room at a later point; i.e.User B adds her friend User C to her Math Room for one session, theroom/board host has the option to allow the other user to see allprevious content in the room and on the board, or provide restrictedaccess to the new user. User B also has the option to erase this contentfor the extra user once they leave again (that is, User C can see theroom/board for that one meeting only) or User B can choose to add User Cto the board's “key users” indefinitely.

v. Another embodiment is that the “peer” or “hierarchical” relationship(i.e. typically tutor-student) is used in establishing settings,permissions, tools and functions. That is, if the session is between atutor and a student, who are signed into the VR application (andtherefore recognized) as such, the tutor automatically holds the rightsto set the settings for that session. This could be done ahead of time,when the tutor is preparing for a lesson, or it could be done during thelesson, in the tutor's user palette.

Each user can access their different rooms in several ways in variousembodiments:

The first way is that users can select their room from the landing pagewhen they first enter the VR application in an embodiment of theinvention. The user is able to create a new room, join a friends' room(by entering the 7-digit code that their friend provided them with) orenter a saved room. All rooms previously created are automatically saveduntil deleted. So the user can select their room from a list in thelanding page. All rooms have a 7-digit code assigned to them uponcreation, but users are also able to assign a vanity name to the rooms,which they can do at the point of creation, when they leave the room atthe end (A pop up asks “do the user want to name this room for nexttime?”) or at any point in time through the room settings in the userpalette.

The other way that a user would navigate to different rooms while in theVR application in an embodiment of the invention, without having tonavigate back through the landing page, is to access their rooms list inthe user palette. From here, the user can see all of their previouslycreated rooms (listed by their 7-digit codes and also by their vanityname, if applicable) and the user is able to navigate directly therefrom their app.

Board as 2D in 3D Space

In this embodiment, the focal point of the VR room is a 2D board whichis linear, persistent and organized. Virtual reality is generally makinguse of a 3D display environment, but when the user is teaching orsharing information with other, it's often better to do things in twodimensions on a board, in a consistent direction (i.e. West to East,just like how the user would teach or share information on a physicalboard). By artificially limiting people to not drawing in 3D space, a VRinformation board of embodiments of the invention make informationsharing more organized. However, the key benefit to being in VR is thatthe user can also work in 3D for a variety of purposes, for example tobuild molecules in chemistry, or to build electrical circuits inphysics, or to visit the pyramids in Egypt. So what is advantageousabout VR information boards of the invention, compared to other apps inVR, is that the entire, ongoing, persistent experience is focused to a2D VR information board, where even the 3D components (modules, models,drawings, shapes, etc.) get stored back on a respective board segment in2D to maintain order and organization over time.

There are several key ways for utilizing 3D space associated to a 2D VRinformation board: (i) flippable board segments and (ii) modules builtinto the board. An additional feature is that in other embodiments of anapplication programmed to provide VR information boards of theinvention, users are not limited to one side of the board segment, orthat they could “pull the whiteboard out”, as if out of a flagpole, andthe whiteboard could continue to wrap around the user, theoretically ofup to 360 degrees all around the user.

Two-Dimensional Representation of Three-Dimensional Objects

Reducing an object's dimensions is similar to converting a color objectto grayscale or B&W or compressing music—information is lost in theprocess. It is important to underscore that the goal in both of theseexamples is to present a minimum possible amount of data to theinformation consumer and still be able to relate with sufficientcertainty the meaning of the original data.

There are two components involved in such conversion processes: theproducer (a compression algorithm or encoder) and the consumer(decompression algorithm or decoder). If the case where both producerand consumer are machines, there are a number of well-known approachesthat are extensively used in communications etc. They can be roughlydivided into two categories: lossless and lossy. The former canmathematically ensure the ability to reproduce the original encodedsignal after decoding, while the latter can only reproduce informationthat is of sufficient quality to convey the meaning of the encodedoriginal.

Lossy compression is used when the consumer does not need the originaldata granularity in order to perform optimally or has some innateabilities to fill in the missing bits of information and reproduce asufficiently accurate representation of the encoded original by thedecoded copy.

The human brain is a prime example of such an innate ability, and thisis why machine image recognition—although making enormous steps in theright direction—still can't compare even with the abilities of a humanbrain.

There are two subclasses of objects: known and unknown. The former canbe stored in a parameterized vector format (i.e. known formula forderiving the object and storing the parameters that fully define it suchas size, etc.) and hence there will be no loss of information.

The latter can be approached in two ways: (a) artificial intelligence(AI)—an AI can be taught what representation the human brain wouldexpect to see from a certain shape to define it with minimum loss ofmeaning, and (b) digital holography—an object can be fully defined bytaking two images (front and back for example).

Regarding the “vector images”, an alternative approach (as the number ofshapes that can be vectorized is limited) is creating a “lookup table”.An example for this approach includes storing the color of each in pixelresource-constrained devices where memory is limited. Instead of storingthe 3-bit color of each pixel, the image is pre-processed and allpossible colors are stored in a lookup table. Then, each pixel onlycontains a small pointer to the lookup table instead of the coloritself. The less colors in an image, the more data storage space issaved.

Similarly, the more 3D shapes that are identified (or predefined), themore a lookup table can be used to store the original 3D meaning. KnownAI-techniques can be used to facilitate the process of identifying 3Dshapes and storing/retrieving original 3D shape information from alookup table.

In the particular case of 3D to 2D conversion, there are two types ofobjects that need to be converted:

1. Known Objects

Known objects are easier to work with as in some cases it is fair toexpect an information consumer (human brain) to already to be inpossession of readily available symbols (e.g. circle with propertint/tone/shade for the sphere, etc.)

a. Known to the information producer: In this case the informationproducer has a ready-made representation of the 3D shape and would onlyneed to adjust parameters such as size, color, etc.

i. Producer has sufficiently high expectations the consumer wouldrecognize the shape/symbol. In this case the producer has previouslycommunicated this 3D shape to the consumer or it is a simple orwell-known symbol.

ii. Producer does not have sufficiently high expectations the consumerwould recognize the shape/symbol. In this case, if there are more thanone possible variations of the representation, the producer would needto guess which may be the most optimal to the consumer or simply sendthe one available, if this is the only option.

b. Known to the consumer

2. Unknown Objects

a. Unknown to the producer: This means that the producer doesn't haveany previous data for the object, which implicitly means that theproducer also has no knowledge of the consumer ability to recognize the2D object. In these cases, the producer will need to create the 2Drepresentation on-the-fly.

b. Unknown to the consumer: Although the previous case covers almost100% of the possible cases, depending on the algorithms (see below),there may be a case where an object does not exist in the producers“arsenal”, but the producer may have sufficient information that theconsumer would recognize a certain symbol, shape or projection.

Based on the foregoing, there are 3 implementation approaches:

i. Library of predefined objects: There will be a library of 3D shapesused in the VR which will have a 2D representation. Participants will beencouraged to use these inside the VR as 2D conversion will be trivial.

ii. Producer frameworks: In case a known 3D/2D shape pair was usedinside the VR, a software framework will produce a 2D shape based onexisting projection algorithms.

iii. AI: It is similar to the Producer framework approach, but the AIwill be taught by means of supervised learning what the consumer expectsto see in case an object is unknown.

Flippable Boards

One of the user settings associated with the board in embodiments of aVR application providing an information board and board segments is thatthe user can choose to “flip” the board, just like the user would see ina real-life whiteboard on wheels in a school. By board “flipping” thismeans that the user can turn the board 180 degrees so that the blankface/side of the board, which was previously on the back side of thewhiteboard, is now showing on the user-facing side. This is an optionalsetting, so the user can either set the board to stay stationary (oneface/side), or they can choose the number of sides/faces associated witheach board segment.

This feature transitions users efficiently from the physical world tothe virtual world without giving them cognitive dissonance, bymaintaining the important things that they need in an informationsharing environment that are in the physical world, but reducing orimproving the components of the physical world that they don'tnecessarily need in the virtual world.

The board flipping feature is useful in many instances, such as:

-   -   The tutor can write theory on one side of the board, and then        flip the board to work on practice problems associated with that        theory.    -   The tutor could write words on one side of the board, and then        flip the board to write their definitions (just like flashcards)    -   The tutor could assign homework for the student on the flip side        of the board.

In embodiments, each board segment has a little grey oval, althoughother control inputs visible to user will be appreciated as capable ofbeing provided in alternative embodiments, on the top and bottom whichacts as the users' handle to flip the board. The action is generatedwith hand tracking, by the user reaching out to the handle and usingtheir hands like they would in real life to flip a real board. Users canflip by pulling down on the handle; but this can flip back again—thatis, if the user doesn't pull the board segment the whole way around,then the board will flip back the way it came as if with real-worldphysics. If the user pulls the board, but not all the way, and then theuser lets go, the board drops and swings back and forth a little bit,and then stops. But if the user flips it all the way over, thetransition is smooth and the flipped board stops quickly so the user canproceed to writing more content and more quickly.

The user is able to choose to keep the board stationary (one-side), ormake it flippable (two sides), or the user can choose any number ofsides that they want. For example the board is able to have 5 sides,like a big 6-foot long, turnable pentagon. To understand this concept,if the user were to look at the cross section of the board, the userwould see a pentagon board that the user could flip and have fivedifferent sides of the board. Essentially the information board is arotatable wheel with 5 discrete board segments in the VR environment.The user is able to add as many sides to the flippable board as theywant, to continue adding space in that board segment without extendingthe room. For example, if a student found one chapter section in theiralgebra textbook particularly difficult, the tutor could continue addingflippable board sections up to 5, 10, 20 segments etc. (creatingcorresponding flappable “wheels” with such number of board segments),for the student to keep working on that type of algebra problem, withouttaking up any additional West-East space in the room, while keeping allof their practice problems grouped together conveniently.

Referring to FIG. 10, examples of a one-sided, two-sided and 5-sidedinformation board 10 all side-by-side are shown. What is also importantto note, is that the thumbnail icon on the top left hand corner of anyboard segment indicates the number of faces of the board behind the facethat the user can see. This prevents the user from losing any content,as they can always see that there is more content on the other side ofthe board.

Modules Built into the Board

When a user does wish to draw something in 3D, like a 3D cube, or asphere, or a cone, instead of just drawing randomly in space, they drawit in 3D in space associated with a particular board segment, so thatwhen they are finished with that 3D item, they store it back on theboard in the correct chronological order. This happens automatically, asthe room is proportioned to be focused around the board; that is, every3D space in the room can be associated with a 2D segment of the board.

For example, User A (the tutor) decides, “I want to explain cubes.” UserA is standing in front of one of the board segments (in VR avatar form).So everything that User A does in 3D is connected to one board segment;the board segment right in front of User A. So each board segment (or inanother embodiment, a marker on the floor where the user's feet are)there is a parallel, or associated board segment. This means that every3D activity that the user create, or session that the user have, iscollapsible into the 2D board, and it is identified with, or connectedto a board segment. So there's a chronology to both the 2D and the 3Dcomponents to every lesson or session. Everything is on the timeline,everything is connected to a board; but this is not necessarilychronological according to when the user created the item with thetimestamp, but instead it is associated with which board segment theuser decide to connect it to; specifically which one the user isstanding in front of at the time of doing the activity. This isimportant as users are able to teleport around in the room, and jumpback and forth to different board segments, to add additional content.For example, a user could be working on the 44th board segment, but thenteleport back to board segment number “20”, to add a piece of 3D contentwhich would then be stored in 2D on the board segment number “20.” Theuser could then teleport forward again to board segment “44” andcontinue working there.

How this works in practice, to continue the example: User A draws arange of different cubes in 3D space. When User A is finished with thatdrawing, the first way that he or she can immediately send the drawingto the 2D board is by clicking a button on his or her tool palette.

Referring to FIG. 11, the second embodiment is that User A 5 can justwalk away, and just by walking away, the VR application folds the 3Ddrawing 110 into the board segment 15. This occurs such that nothing isever lost in 3D space; as soon as the user is not working with the 3Dobject (indicated by them walking away), the system stores the 3Dcontent in 2D drawing 120 on the board. When a 3D object is stored onthe board, the system puts a little bookmark 115, which looks like alittle lever sticking out of that board segment in three dimensionalspace, that is a visual cue: “There's more to it than just this twodimensional board segment, there's a 3D drawing the user can pull outand push back in.”

The third embodiment is that User A can pick up the 3D object (because3D objects can be picked up and interacted with in VR as if they arereal objects) and hold it against the board. When it is being heldagainst the board it flashes to indicate that it is about to turn 2D,and then the 3D object is replaced with a 2D “sticker” representing the3D image, which is then “stuck” to the board in 2D. The user is thenable to tap this sticker to cause it to pop back into 3D in front of theboard. The object will otherwise remain as a 2D sticker on the boardwhere it was placed until brought back into 3D by the user.

Beyond the obvious and unique benefit of storing all VR contentchronologically, this feature has two additional benefits.

The first is that storing content in 2D saves bandwidth because theuser's device cannot support having all of that 3D stuff always loaded.So it basically offloads the 3D objects to not be loaded unless the useris in front of that board or near the front of that board.

The second benefit it has is that: when the user is trying to navigatedown the line, the user can go down the board segments, the user can godown the board on the wall and not have all of the 3D objects, modules,games, etc. in the user's way as the user is walking through it.

Two Sides to the Board

In a typical VR environment of the invention, users are only going tostand on one side of the board. However, in other embodiments, users canmove to, see and use both sides of the board. In the primary embodimentof the VR application, people are generally kept on one side of thevirtual 3D information board, so that it makes it simpler to followwhat's going on in a linear fashion. The user could flip the board andsee the other side, but the user (in the user's VR avatar form) stays onone side of the board. In one scenario the user is on the South side,the user is looking North at the board, the user cannot go to the Northside of the board, the user remains on the South side of the board andthe user just flips the board if the user want to see what is on theother side, and work with what is on the other side.

However, in another embodiment of programming a virtual realityenvironment with an information board, users could “step through” theboard and then work on the opposite side of the board from the oppositeside—essentially doubling their working space.

Wrap Around a Whiteboard which can be Pulled out of a “Flagpole”

Another embodiment of the virtual reality environment, such as a VRclassroom, is that of a “wrap around” information board 10, which ispulled out of a flagpole-type structure by the user. The user can, forexample, pull a whiteboard out of a flagpole-like structure and thewhiteboard itself is flexible, but then it snaps to being flat, so theuser can actually draw on it, and the user can pull it out, pull it outsome more, pull it out some more, and then the user can pull it out evenfor a full mile or more if the user 5 wants that long of an informationboard, and the user can later put the board back in to the flagpole-likestructure to save space. There are several embodiments that thistechnology can be implements:

i. A flagpole whiteboard for extending the information board 10 is shownin FIG. 12. The main way for the user 5 to extend the information board10 is to grab on to the end of it, for example at the far West side ofthe board, on the end, and to pull on it to continue extending theboard. The board can even be pulled around the user in a full circle(360 degrees) if the user continues to pull the board around them asillustrated. In embodiments, a “Curved UI” Unity Asset Store plugin(published by Chisley) is used in order to enable a curved userinterface and prevent distortion of the content on the whiteboard.

ii. Flagpole whiteboards that are pull-out, flexible, snap-to-positionwhiteboards are also able to be used as another way to get more boardspace without adding additional segments from East to West as shown inFIG. 13.

For example, a tutor as a user 5 could say “I've got a big concept Iwant to talk about with a student today, but I don't want to mess up ourchronological, organized board segments of the main information board 10while I explain the details of this concept. I'm going to stick aflagpole between these two board segments (or alternatively, just pullhorizontally out of the East, West board).” In this example, the tutorcan pull a flagpole board 130 from the main information board 10 outtowards them, i.e. towards the users or perpendicular to the mainEast-West board. Then when the tutor and student are done with thatadditional content, they can slide the additional board 130 back intothe main board 10. While the main board runs horizontal from East-West,when the user pulls a flagpole board segment 130 out South, towards theuser, it should preferably be a horizontal board as well.

Some embodiments of use-cases where tutors and students benefit fromthis tool include:

-   -   Users can practice problems without taking up a huge amount of        space on the VR information board. This is important because        users have to be able to teleport or travel down the length of        the board over time to be able to review content. Users want to        be able to do this efficiently, so any repetitive or messy tasks        which would use a lot of space (such as practice problems) could        be put on a horizontal board so as not to take up too much space        in the timeline of the whiteboard. They can be stored for easy        access if the users want to access them again in the future, but        otherwise they are out of the way.    -   Users are able to bring in a homework PDF or a worksheet and        they often don't want to interrupt the flow of the content on        the board with these additional worksheets. Instead, they are        able to put the additional content on a flagpole or        perpendicular board segment, to be used and then stored for        later.    -   When a user comes back to a board section later when revising        the user's work, the user may want to add additional notes, or        link to another theory, or draw an additional diagram to explain        a concept now that they now understand better, or insert a        module. They might want to do this without interrupting the flow        of the main board, so they could set up a flagpole or a pull out        or a perpendicular board here.    -   If a user thinks of something they need to study later, but        don't have time to do in the current session, or if the tutor        sets homework for the student to go back to, they would set that        up as a perpendicular board as a placeholder for when the user        come back to that in the future.

iii. Another embodiment is that the main landing page for a VRenvironment application is able to be designed as a series ofinformation board “flagpoles” with the name of the room on them, such asthe vanity name that the user has created to recognize that room whenthey go back to it over time. And when the user chooses that flagpole itunwinds to create the user's board of the designated length that theuser has either created, or to the default minimum length if it's a newroom, and when the user is done with it and when the user exit the room,it zips back into a flagpole and then the user sees all the user'sflagpoles again in series again in the landing page.

Augmented Reality (AR) Infinite Circular Room:

Another embodiment provides an extension of a 2D-wraparound whiteboardfor Augmented Reality (AR). When using Augmented Reality (AR), the usercan't extend the board out infinitely, because the user's house orbuilding where the user is using the AR has walls, and the augmentationin AR is simply overlaid over the physical world.

In embodiments, the AR boards are able to be wrapped around in a fullcircle, that the user is standing in the middle of. The board wrapsaround the user in a circle and the user can then just grab or pinch thewhiteboard around them (using hand tracking or controllers) to move toall the boards around them in space; that is, the circle spins aroundthe user as they manipulate the board with their gestures and controls.For example, the user can spin the circle to go back to chapter 1 orforward to chapter 10, and the content will spin around constantly, likeit is unfurling.

This occurs with the use of a seam where the new content enters, whichis positioned directly behind the user in the circle, to give theimpression of infinite content spinning around the user in a circle.

Spotlight Board Segments Above the Users Head

Referring to FIG. 14, when using a virtual whiteboard, in embodimentsthe user 5 can look up at any time and above their head is a circle 140of board segments, preferably as “thumbnail”-type board segments forproviding information board navigation for the user 5. When the userlooks up, one of the board segments is highlighted: this will be theboard segment 145 directly above the user's head, which will correspondto the board segment 15 the user is standing in front of in the VRenvironment. Referring to FIG. 15, when user 5 spins their body,different board segments highlight, since the head-mounted display knowsthat the users orientation on the ground corresponds to a board segmentin the circle 140 of thumbnail segments above their head, and so thespotlight section changes depending on the way the users body spins.When a thumbnail board segment 145 spotlights, it is preferably madelarger so the user can see that particular board segment more clearly.As the user just spins the user's body, or in another embodiment, theuser could also reach up and use the user's hands to spin that littlecircle 140 of thumbnail board segments and a different board segment isspotlighted.

Referring to FIG. 16, once the user has found and spotlighted the boardsegment 145 that they want to jump/teleport to, the user can reach upwith their hands and pull that spotlighted board segment 145 down, fromwhere it is floating above the users head. The pulled board segment willthen be right in front of the user. For example, a board segment may bepulled down, and extend all around the user (e.g. in the circle aroundthe user, or in another embodiment the linear board around the userextending infinitely) and also with adjacent board segments. Forexample, a board segment 15 of interest along with the other adjacentboard segments (ordered before or after the segment of interest such asillustrated as board segments “44” (before), “45” (segment of interest)and “46” (after)) in the continuous information board 10 are alldisplayed around the user and all of these segments ready to beinteracted with by the user.

The key benefit of such embodiments is that the user can see many moremini thumbnail board segments when they are in a small circle above theuser's head, and the user can scroll through them more quickly, becausethey are much smaller and more compact than the full-size board segmentsaround the user at chest level. The spotlight circle above the user mayhave 5× or more as many board segments than if in full-sized boardsegments around a user, since they are tiny thumbnails that only openwhen in the spotlight region so the user can see them better as the userspins the user's body around.

Spotlight Board Segments with Additional Features and Pages

Another embodiment of the foregoing technology includes adding pagescomponents to in VR environment with the information board segments thatare not information board segments. For example, a circle of pages maybe displayed above user's head that can be a combination of informationboard segments as well as non-board segments, such as document files,applications and application windows, and the like.

The non-board pages may be open for productivity, such as Excel windows(Microsoft Corporation, Redmond, Wash.) or Chrome tabs (Google LLC,Mountain View, Calif.), when the user is multitasking the user'swindows. The user looks up and the spotlight circle overhead spotlightspages and then the user use the user's hand to reach up and pull down arespective page that the user want in front of the user, in the manneras described with respect to board segments above a user's head.

If a user, for example, has twenty things open on his or her computer,he or she may want to have access to all of them in VR or AR, and alsowants to be able to pull the important ones (primary items) around her.In embodiments, the VR environment is programmed so that the user ispresented a graphical user interface to set up his or her ‘primaryitems’ in the VR environment. The user can then look up, spin his or herbody or hands t and find the pages/items/files, and then pull them down.

Another embodiment of this concept, is that instead of looking up, theuser looks down, and the VR environment is programmed to display thepages/items circle down around the user's feet, and the user would reachdown or drag/swipe things up into their primary items.

Referring to FIG. 17, an Excel sheet is the primary board page 175 infront of the user 5, and can also be seen as a thumbnail page 171 in thecircle 170 at the user's feet.

Referring to FIG. 18, when the user 5 turns their body clockwise (asdepicted in the image below), the next most clockwise board segment (ornon-board page in this case), is highlighted. In this example, it is animage file.

Referring to FIG. 19, after turning their body and highlighting theimage file, the user then chooses to pull up this image file to theirmain primary board, moving the excel sheet to the next board segment(out of view).

The primary circle is highly important items that the user can interactwith (the normal, full-size board segments described earlier), and thisoccurs typically at chest level. The secondary circle of smallerspotlight board segments is above the user or below the user, or bothabove the head and the feet, if the user has lots of tabs and pages andchooses to set both up. These secondary pages look similar to the ‘dock’on the bottom of a laptop operating system screen, where icons on thedock become bigger when the user hover over them. Except in embodimentsof the invention in a VR environment, a circle above the user or belowthe user is displayed and the user can flick/swipe/grab things and pullin from the circle (or similar VR docking mechanism) or put them back inthe circle (or similar VR docking mechanism).

Board Segments Projected on a Physical Wall

Referring to FIG. 20, in some instances, a user 5 may want to use anactual wall in their house to write up against as if it were aninformation board, or they may want to use the wall as the orientationof their board in the VR room, like a physical whiteboard.

This is also helpful if there are multiple users, either physicallytogether or virtually, who all want to collaborate together on theboard. For example, there may be a situation where there are 5 studentstogether in the physical room, and another 15 students who want to workwith them virtually in the same virtual room. A board can be projectedon to the wall for the 5 students in the physical room for them to seewhen they have their VR headset on. So the 5 students in the room withheadsets on would see the board taking up one whole wall of the room,and they would understand the spatial boundaries of the physical spacethat they are standing in.

With the projected board, two additional embodiments can be implemented.One is in the VR environment where the students are fully immersed inthe VR world, and another is in an AR environment, where the studentscan still see the physical room around them, but they can also see theVR room and everything in it at the same time. The additional 15students who are meeting with them virtually wouldn't see the physicalroom boundaries, as they are not in the same physical room, but theywould see the same board projected in VR that the 5 students in the roomare seeing in either VR or AR.

In the physical room (or in the virtual room for the other students),the students can change the content showing on the projected whiteboardscreen in multiple ways:

-   -   by flipping the board, as explained in the section describing        Flippable Boards    -   by using the Spotlight method disclosed previously    -   by swiping the scrolling bar on the bottom of the board, which        would swipe the board across to the next board segment, which is        now ‘projected’ on the wall. The users can flick the board to        the side using their hand tracking gestures (or controllers, in        another embodiment), and the board changes to the neighboring        board segment.    -   by grabbing the board to move it. In this gesture, the ground        stays where it is, but the board itself can be slid left and        right and up and down. This allows the user to access more blank        space on the board to continue writing, or to access previously        written content, by grabbing the board with hand tracking and        pulling the previous content into view. In embodiments, the        board cannot be moved closer to the user through this method, as        it is locked onto the axis of the physical wall. If the user        tries to move the board closer to the user, the system will        bring the user closer to the board, but the board remains        stationary and is not moving. In this instance, the other user        will see the user slide the board left and right and up and        down, but the other user would not ever see the user pull the        board closer to the user (and thus closer to them) because        that's not allowed within the constraints of the navigation        controls. Instead, the other user would just see the user move        closer to the board or farther from the board.

Another embodiment is that the student can walk up to the physical walland there would be a left and right arrow projected on the actual wallthrough AR or VR. When the user then touches the actual physical wallwith their fingers, which are tracked with hand tracking, the headsetknows the user want to press left or right (which are forward or backbuttons) to change the board segment to the neighboring segment. This ismuch like the buttons used to extend the board segment as describedpreviously.

The key difference in this embodiment, as compared to the infinitewhiteboard embodiments above, is that the students are enabled to use ARas an option, as well as VR, and that at any time, there are only alimited selection of board segments projected in the room, compared toin VR when the room appears to be infinite. This means that instead ofthe board going through the user's wall somehow, outside the user'shouse, which does not make sense when in AR, the students can stand nextto a physical wall together in a room, and see each other through AR andall see the same board projected on the wall.

This means that there would be no need for a physical whiteboard, if aboard could be projected straight on the user's wall (which the usercould see when the user has a headset on).

Another embodiment includes multiple students working together in onephysical room, and each student could have their own board segmentprojected onto a different wall in the room. For example, Student 1takes the north wall, Student 2 takes the south wall, Student 3 takesthe east wall and Student 4 takes the west wall. Each student can workindividually on their own board segment, while in the same space as theother students. The students could also walk to each of the otherstudent's parts of the room and read what they have each written ontheir boards, and interact with them. Users therefore could use all 4sides of a 6 ft×6 ft (or other dimensioned) room (see FIG. 20). And all4 students can see all 4 boards. Another embodiment of this technologyis to use however many walls are available in any given room. If theroom is an unconventional shape, like a triangle for example, with threewalls, this doesn't matter, the user can simply place the board segmentsall over the walls that the user do have, so the user can see more than1 board at a time and walk around the room to the different boards.

3. Room Customization & Selective Parity

By nature of being in virtual reality, many components of the room areable to be customized and changed. This includes but is not limited to,the background of the room, the board itself, the floor/ceiling/walls(if any selected), and any furniture in the room. Currently, VR spacesallow individuals to experience different worlds and scenarios, but theexperience is wholly singular. Technology created a situation where anindividual is able to access many worlds, but the individual issimultaneously isolated. There is not a known conventional solution thatmakes VR simultaneously individualized, yet collaborative. This is acrucial missing link, made especially evident in any arena of learningor training.

Improvements offered by embodiments of a programmed VR environment ofthe invention, include creating a VR space “classroom” that incorporateshuman knowledge of the individual to create an individualized roomexperience that can also be collaborative with others (e.g. aninstructor and/or other students) in the VR “classroom” or space. Ateacher, for instance, can set up certain sequences of environments,customized for each student (as he or she knows is needed for individualneeds of students), that addresses students' needs. An instructor canadapt the virtual space for each student/user (this can be done for jobtraining) as well. The VR experience is being moved from a purelyindividual to a collaborative experience. The teacher can be with thestudent in the same virtual reality world, or one teacher can be withmultiple students, manipulating the world to best suit each student'spreferences and needs. These different worlds could be based primarilyon personal preferences, or they could be more specifically tailored toa student as required for their learning styles, skill levels, health orlearning difficulties. For example, some students who are easilydistracted might be suited to a more simple background and nospatial/background audio, which would not cause as much distraction.Students with reading troubles may benefit from the font size beinglarger on all textbook files being imported into the room, which theteacher could set for all sessions with that student, ahead of time.Students who have hearing difficulties could have real-time closedcaptions automatically on by default for each session. A student who iscolor-blind, for instance, may have certain colors automatically changedfor them (e.g., a red-green color blind student will always seered-color notes in blue instead), or have patterns built into colors.

Different Adaptable Spaces

Some contextual factors of the virtual room (e.g. background, floor,furniture, whiteboard/blackboard) can be changed in embodiments. Thesechanges can occur in VR for one user but not the other. For example, asshown in FIG. 21, two users 5A and 5B are able to customize theirbackgrounds differently: e.g. user 1 (5A) can set their background 512to the room as if they at the beach, while user 2 can set theirbackground 514 as if they are in a classroom. These two users cancollaborate while each experiencing different backgroundssimultaneously. For other factors, the two (or more) users mustexperience the same information on the information board 10simultaneously, such as the content on the information board, the colorof the content on the board, and the color of the pens each user isusing. In general, the rule is that the content must be consistentbetween users, but the context in which they experience this may change.This is integral for tutoring because when referring to content on theboard or content of the lesson, there cannot be any confusion betweenusers. For example:

-   -   User 1 is in a classroom, while user 2 is at the beach, but the        whiteboard and everything on it, including the practice problems        they are working through, are consistent between users    -   User 1 chooses to change their classroom background to a forest        and change the color of the pen in their hand from blue to red.        User 2 stays on the beach, but also see's user 1's pen change        from blue to red.    -   Both users are working on the whiteboard as the information        board. User 1 decides she would prefer to work on a chalkboard        instead. The board experienced by user 2 remains as a        whiteboard, while the board experienced by user 1 changes from a        whiteboard to a chalkboard. All of the writing and files, etc.        on the board retain their color, size & all other features, and        user 1 is able to continue collaborating on the board. User 1        will now see the board as a chalkboard, while user 2 continues        to see the board as a whiteboard.    -   While working at a table, user 1 prefers a wooden desk while        user 2 prefers a glass desk. Everything on the desk remains        consistent for both users

These customizations are also able to be personalized based on a dynamicinput which comes from a source external to the VR headsets, applicationand VR server computers:

One example of dynamic input is weather input: a user could have theirsettings such that they appear to be on the beach, and the beach scenechanges based on the weather. If the weather is raining in thereal-world, then their beach scene will also appear as raining; or sunnyif it is sunny etc. The weather input comes from an external weather Website, Web page, weather station hardware and software, etc.

Another example is that the user's background setting could be variablebased on the time of the day. If it is the morning, the sun could berising in the distance, with the sound of birds chirping. In theevening, the moon could rise and the audio would be more silent.

These customizations can also be used according to research andparticular student's needs. For example, changing the color of walls tobe blue is supported by research that it's more calming, and there isresearch to show that people remember more when they are in a greenspace. The benefits of VR are that each student can have a differentcolor of wall in the classroom, and additionally, this can be trialedand experimented much more quickly as compared to assessing in thereal-world (no need to actually buy paint and re-paint to determineresults in VR). These customizations provide and are based onpersonalized feedback, as the VR application will learn, throughcommercially available AI, which environments are most effective on aper-user basis (e.g. by the results of peer users, or by the currentuser's results (such as test scores, attention assessments, and otherresults) using different environments).

Environments and Specialized Environments

In addition to the basic environments, there are also specialistenvironments which are custom purpose environments for completingdifferent tasks. A programmed VR application of the invention can haveseveral specialist environments and will continue to develop morespecialist environments with specialist partners, universities &colleges. Example specialist environments include but are not limitedto:

-   -   tutoring room—like a breakout room or private room for a tutor        and student    -   classroom—for multiple (more than 2) people to collaborate        together    -   electricity room for physics—circuits room where the user can        see a circuit completed in 3D, while the equation is being shown        at the same time.    -   molecule room—a chemistry-based room with molecules or processes        (like the Kreb cycle), where users can have an activity and/or a        matching game or something other little simulation or visual aid    -   library—where the user can read all the Openstax (Rice        University) books and sit quietly. This library room has a        beautiful, modern library aesthetic, with all textbooks the user        needs are accessible right there (through Openstax integration),        also public domain books may be kept here too. In this room,        users take voice memos, highlight content & take notes which        they export to their computer or store in the VR app to revise        later. This room is the ultimate in immersive reading, with page        flipping in zero gravity, and definitions on words in textbooks        can be experienced, but in VR when there is a recognized place        or object, the user can bring that element up in 3D to interact        with it or explore it.    -   accounting statements room—users can select from various public        companies and the balance sheet an income statement and cash        flow statement pop up; and then the user can select the next        company and then those statements pop up instead.

Modules in Adaptable Spaces

The classroom itself is to be a very foundational space (infinitewhiteboard with a user palette to access all necessary tools forlearning) but the users can alter this space to teach specificapplications. Examples:

A mirror room or space for chemistry to be able used:

-   -   With the molecular bonds, being able to flip them and draw        vectors to show dipole moments.    -   Ability to show mirror images for isomerism. Letting the student        look on one side of the glass then walk over and show the other        side of the glass. Then shrink the molecules to show they are        not superimposable    -   Ability to go to job sites like construction and show how cranes        work for physics and test track to show velocity    -   Ability to dive into problems to see the conductibility of        solutions for chemistry    -   Show how attractions (IMFs) work on the compounds studied.    -   Virtual car where instructor shows student user how to change        the parts

These tools can be integrated as APIs, SDKs, or raw source code asneeded.

Sit/Stand Customizations:

In embodiments, a VR environment of the invention is programmed so thatusers can use the VR environment and information boards while bothsitting and standing. The transitions between these positions and themulti-user components of the feature are smoothed through several uniquesettings outlined in the subsequent description.

Sitting/Standing Transition and User Interface and Settings

When a user's headset detects that they are sitting down, (suchdetection occurs automatically), the virtual whiteboard rotates frombeing vertical, at 180 degrees (like a normal wall/whiteboard), to beinghorizontal at 90 degrees, as if it's like a desk.

In order to capture a baseline height of each user, this is oftencaptured by most head mounted VR displays upon set up, however, this isalso verified in a programmed VR application of the invention by askingthe user at the start of the first session of application set up: “Isthe user sitting or standing right now?” in order to determine theheight of the user at standing, and use this as the baseline to thenmake automatic assumptions later in the VR application in an embodimentof the invention. Other alternatives to detect the height the firsttime:

Sitting and standing behaviors: if a user is moving around a lot then itis most likely that they are standing, and can be treated as such.

The external cameras in a head-mounted display (HMD) can see the floor,or other surroundings around a user, providing information on the user'sstance. For example, if the headset detects that a user's headset is 5ft off the ground or the closest surface, then that user is assumed tobe standing.

For example, a user is holding a marker in their hand and is standing upby a whiteboard, and the orientation of the whiteboard is 180 degreesvertical (compared to the user's desk which is a 90 degree, horizontalorientation). The user then gets tired after standing for a while duringtutoring, so the user chooses to sit down. As the user sits down attheir desk, their headset measures this change in height andpositioning.

There are several options once the user has sat down, as to whether theboard reminds vertical, automatically transitions to horizontal, orsettles somewhere in between horizontal and vertical. These settings areoffered to the user the first time they transition from standing tositting in the VR application in an embodiment of the invention, andthey can choose to save their preferences, or re-select them each time.Once they have set their preferences, the user can also change theirmaster settings in their user settings. The different embodiments ofthis feature are outlined as follows:

i. One embodiment is that the user has sat down, but they still use theboard vertically at 180 degrees (or approximately thereto in verticalorientation). They can sit down in the chair or on the floor and holdtheir arm up and continue to just draw in the sky or against a physicalwall in their house, at 180 degrees vertical.

ii. Another embodiment is that once the user has sat down, they are ableto change the board from vertical to horizontal by manually ‘pulling’the board to the horizontal plane. This action is done by virtual“handles” in VR which are always visible in one embodiment, or onlyvisible when transitioning between sitting and standing in anotherembodiment. These handles allow the user to actually grab the board andpull the board to vertical, horizontal, or anywhere in between, in 5 or10 degree increments, to choose where they want the board to be set.

iii. Another embodiment is that the interface automatically recognizesthat a user has changed orientation, triggering a pop-up which says: “Wesee that the user has sat down. Would the user like to reset the user'sboard orientation?” At this point in time, the user can select “remembermy settings for next time” in which case that will become their defaultsetting for the next time they transition between sitting and standing.

The difference between a vertical and horizontal board and the way ittransitions between these two orientations is demonstrated as shown inFIG. 22.

Different Horizontal Board Options

When the user chooses to bring the board onto the horizontal plane,there are also several options that they can choose from. Preferably,proportions are maintained between the height and width of each boardsegment. Whether the user chooses horizontal or vertical, the horizontalboard will maintain the same proportions as the vertical board. That is,if the board is two miles long before it is rotated to horizontal, itremains two miles long after it is rotated, and the board width remainsequal before and after the rotation, to keep proportions maintained. Ifthe user chooses to make the horizontal board half the height (which isan option in the user palette, to allow the user to reach across theboard more easily), this also readjusts the width of the board(shrinking the content proportionally) to maintain the proportions andtherefore readability of the content on the board. An example is that aboard may initially be 18 ft long by 3 ft high. When the user brings itonto the horizontal board orientation, this is adjusted to 6 ft by ft,by shrinking all the content (and the board itself) by ⅓, in thehorizontal embodiment of the board.

Movement Along the Horizontal Desk

The user can grab the board (which holds all the board content that waspreviously vertical, which is now sitting either horizontally in space,or on the virtual “table”) with their hands and they can execute a “graband pull” motion in VR, which will slide the board (with its content)across the desk (horizontal space in front of the user) in the directionthe user pull it. The user themselves wont slide or move, but the boardand its content will slide relative to the user in VR.

The grab motion requires the user to touch and squeeze the boardcontent. In more detail, this gesture sees the user reach through theboard; close their fists around the board, which will activate a visibleglow to the user's VR hands, indicating that the board has beensuccessfully grabbed; finally the user is able to pull the board in anydirection they please.

The board itself is fixed to the horizontal (or angular, if set on anangle) plane, and thus can't be pulled completely upwards or off thedesk/plane by the user. It can only be slide side-to-side to enable theuser to see more content. In order to change the angle of the boarditself (i.e. from horizontal to vertical orientation, or somewhere inbetween), the user will need to use the process outlined in 4A above(sitting/standing transitions and user interface).

Different Embodiments of the Vertical Table

If the table becomes horizontal, there are different ways that this canbe experienced by the user, which is selected through the customizationsection in the user palette:

i. In one embodiment, the board simply transitions from vertical tohorizontal, turning into a floating, horizontal board. This board is notplaced on a table, but instead is just floating in space, just like whenvertical, except now flipped onto the horizontal plane.

ii. Another embodiment is that a desk, a wood or metal or glass (or anyother type of customizable desk, which can be brought into the VRapplication through 3D modelling integrations such as Google Poly), canbe enabled. This gives the user the experience of sitting at a desk,with the board running along the desk as if it were a whiteboard laidout on a desk. The desk is placed automatically underneath thehorizontal board, and the board is on top of that desk, like a huge rollof paper that's been unrolled down the middle of the desk.

Referring to FIGS. 23A and 23B, there are two different embodiments ofthis virtual desk which are available in a programmed VR environmentapplication of the invention, which the user can toggle between based ontheir preferences for the style. The two options include:

In FIG. 23A, the desk is automatically the same distance long as theboard itself, which means that in the instance of a very long board, thedesk would be seen to extend into the distance.

In FIG. 23B, the board is as long as the content that it holds, but thedesk/table remains a set size (which is able to be specified by the userin their tool palette if they wish to customize it, otherwise 12 ft longby default). The board is seen to fade into a gradient at the end of thedesk, indicating that there is more content but the user must scroll tosee it. The user can scroll by grabbing the board with both hands (reachthrough and squeeze gesture) and then pulling the board to either side.

Setting the Height of the Vertical Table

The table (or the horizontal board without a table) can be set to avariety of heights, which the user can specify. There are several wayswhich this setting can be edited, which includes the following options:

i. If using controllers, the user can put the user's controller on thetable and the controller will set the height of the table, such asoccurs in an Oculus headset (or may be implements in similar headsethardware device) to set the height of the floor. If using hand tracking,then the user can rest the user's hands on the table and this will setautomatically. This occurs due to the external cameras on a HMD whichare able to detect the nearest flat surface.

ii. If the table is set at the correct height automatically, the usercan access this setting in their user settings and reset the tableheight. When they choose to do this, they receive a popup message whichreads: “Please rest the user's hand on the table.” This allows the userto manually reset the height.

iii. The VR program can detect the height of a table based on where auser is writing with hand tracking. As the user is writing, on aninvisible horizontal plane, the system can detect where the user desirethe height to be by the angle, height and positioning of the user'swriting. The height of the table is automatically set to respond to auser's hand positioning.

iv. One embodiment designed to merge the real-world with the virtualworld is a standing desk in virtual reality where the user uses a handcrank on the edge of the table. That is, the user is able to walk overand crank the handle to raise and lower the desk, up and down to adjustthe height. The hand crank is operated with hand tracking, and also withcontrollers if preferred.

v. Or there is also a setting in the user palette in system preferencesto adjust the height, emphasize the height, up and down.

vi. Using hand tracking, there is a gesture, such as raising andlowering fingers with the palm facing upwards, held out in front of theuser, which moves the board up and down slowly. These gesturepreferences may also be altered in the user palette system preferencesettings.

Angle of the Board

The angle of the board, whether it is vertical or horizontal orsomewhere in between, can be adjusted by the user. There can be manydifferent iterations of the user's board or table beyond just vertical(180 degrees) and horizontal (90 degrees). A board segment 15 at 45degrees is shown in FIG. 24. Some of the implementations for thisfeature are:

Using the table like a drafting table, which is not a flat table, butinstead on a slight angle.

If the user want to lie on the ground, or if the user want to lie on theuser's back on the floor with the user's hand, no marker in the user'shand or a marker in the user's hand, doesn't matter. It's just trackingthe user's hand. The user could literally lie on the user's back forsomebody who's incapacitated or has a bad back problem, and the boardcould be either directly above the user, or on a slight angle, whateverthe user prefers.

The options for how the table angle can be adjusted in the VRapplication include:

i. The user manually adjusts the angle in virtual reality, for exampleby pulling a handle out of the board and using that handle to turn theboard on an angle, which is detected automatically in 5 degreeincrements.

ii. The board constantly gets smarter and better based on how the usersare drawing, the position of their heads and controllers or trackedhands, and their user settings from the last time they used the VRapplication in an embodiment of the invention to draw.

iii. There is an interface which allows the user to type in theorientation (in degrees) that they want the board to be positioned at,in the user settings on the tools palette. So the user could type “65degrees” and the board automatically shifts to that position.

When the user first enters the room, the board is pre-set as awhiteboard, oriented at 90 degrees vertical. This is the defaultposition whenever a new user enters the VR application in an embodimentof the invention, or a new room is created. This default position can bealtered by the user. Once set by the user, this preference is maintaineduntil adjusted again by manual input by the user.

Customizing Avatars and Avatars in Multiuser Mode

Users can Customize Own Avatars

Users can customize their avatars and how their avatars appear andinteract with other user avatars too. Some of the changes one mightchoose to make to their avatar are as follows:

i. Change clothing

ii. Change skin color

iii. Change height (for example, if User A is a 6 foot 2 tutor, and UserB is a 5 foot 4 student, User A might choose to shrink his or her avatardown to her size to make User B more comfortable. This may be pre-set byUser A (as the tutor), that he or she is always automatically updated tothe same height as whichever student is in the room with him, or thismay be done manually as soon as another user enters the room

iii. Age: students or tutors may choose to change the age of theiravatar if it makes them more comfortable.

iv. Pitch of voice to make other users more comfortable.

v. Ability to change size (the person themselves) both to make otherusers more comfortable, but also for practical purposes relating todifferent modules, such as to explore large and small spaces. Forexample, if the user are working in a module where the user areexploring termite hills, the user can choose to make the user's avatarshrink down so that the user are tiny compared to the scale of what theuser had previously seen in actual size, and then the user can walkthrough the termite hill to get a better understanding. Another exampleis traveling through a blood vessel.

vi. Change the gender—e.g. if User A is tutoring User B and User B feelsmore comfortable with a female tutor avatar, User A makes his or heravatar appear as female instead

Users can Customize how they View Other User's Avatars

Editing the user's own avatar in VR is possible, but what is also uniqueto our application is that users are able to choose to customize howother users' avatars appear to them as well. This is helpful in thecontext of tutoring because students can learn better when they arecomfortable with their tutor, which includes the importance of thetutor's appearance.

Changes in the appearance of other user's avatars will only occur fromthe perspective of the user who has made the changes. That is, if thereare three users in the multiuser room (one tutor and two students, forexample) and Student One changes the appearance of the Tutor, theStudent Two will not see these changes, and instead will still see thetutor in their original/default appearance setting.

The tutor will also be able to see in their user palette how eachstudent has chosen to view each user's avatars, which may help the tutorto set their default avatar appearance in the future. For example, atutor user may prefer to set an older-appearance for their avatar whentutoring mature students.

Users are able to change several settings relating to the other users'avatars through the settings that are found in the users tool palette.Some of these settings are as follows:

i. Avatar transparency: if the other avatars are obstructing the user'sview of the board, the user can toggle their avatar transparency on oroff so that the obstructing users just appear as an outline, as depictedin the drawing below. In other embodiments, the user could appear as ashadow, or a light orb, or another non-obstructive representation.

ii. Ability to change the age of the instructor. This setting is helpfulin the context that older learners tend to have preferences forinstructors who are their age group or older. Therefore, users canchoose to change the age of both their avatars, but also of the otheravatars they are working with.

iii. Ability to change the voice of the instructor, specifically thepitch and tone of the tutor's voice, to a voice that the user prefer tolisten to. Accents might also be changed.

Iv. Users can change the skin tone of their own avatar, or of the otherperson (typically not before the user has even seen them in thatsession). These settings can also be edited at any time during thesession, after it has begun, but also can be set beforehand tostreamline the initial introduction between users. These settings can beset manually, or the user can click a box that says “change the skin ofthe teacher, to be the same color as the skin I have selected for my ownavatar’. An alternative embodiment might provide the option to: “Checkthe skin of all the people in the room to the same color as my skin.”

Avatar Height Parity

In another embodiment of the VR application, and with reference to FIG.25, instead of users choosing to customize the height of their own, orother user's, avatars; the application automatically senses the heightof each user and sets the board, room and other user positions relativeto this to make it more comfortable for both users. Three options forhow this height can be set:

i. Based on the height of the tallest user; that is that the taller usersees the world as normal, but the shorter user has their experienceadjusted upwards (they are now seeing as though they are as tall as thetaller user).

ii. Based on the height of the smallest user; that is the smaller usersees the world as normal, but the taller user has their experienceadjusted downwards (they are now seeing as though they are as short asthe shorter user).

iii. An average of the users' heights: both users' 5A and 5B experienceis adjusted proportionally in each required direction such as to putboth users on the same plane for viewing and experiencing an informationboard segment 15 in VR. That is, the taller user experiences the virtualroom as if they are slightly shorter, and the shorter user experiencesthe room as if they are slightly taller.

When these adjustments take place, the relative height of the board anddistance between the board and the users must be accounted for and keptproportionate. Furthermore, the user's relative arm lengths for usingthe board must remain persistent, as the user needs to be able to expecttheir arm to be the correct distance away from the board at any giventime. In order to adjust for this, taller users are positionedever-so-slightly further back from the board by default, so that theirlonger arms still feel proportionate when they reach up to write on theboard, but they appear to be the same height as the users around them inthe VR room.

Another way that this can be built, in another embodiment, is that theboard (and the entire room) ratio-scales based on the height of theuser; which means that shorter users would just have slightly fewerpixels in their display than taller users.

Avatar Sit/Stand Parity

Avatar sit/stand parity is a setting which can be toggled on or off inthe user palette of the VR application in an embodiment, which enablesthe orientation of all other avatars in the VR room with the user toappear as though they are in the same sit/stand position as the user'savatar.

When avatar sit/stand parity is turned off, then everyone in the room isdisplayed in the position they actually are in (that is, if user 1 issitting and user 2 is standing, for example, then both users appear assuch). If the setting is turned on to create parity between users'positions, then the different embodiments of this are explained below:

-   -   If user one chooses to sit down, then everyone else in the room        also appears to be sitting down, whether they are actually        sitting down or not. That is, the other users' avatars are        adjusted in virtual space to appear to be at the same height,        from the perspective of each user. This means that in the        example of User A as the tutor & User B as the student, that if        User A is sitting and User B is standing, User A sees both        himself/herself and User B as sitting avatars, while User B sees        both her own avatar and User A's as standing. The actual        positioning of each avatar doesn't matter because the virtual        board is unchanged and is showing to each user, so input can be        edited regardless of avatar positioning.

Referring to FIG. 26, from both User A's (5A) perspective (while he orshe is sitting) and User B's (5B) perspective (while he or she isstanding) they appear at the same position on the board segments 15.They are working together on the same board segments, with the samecontent on the board for both User B and User A.

User A is lying on his or her back, User B would appear to be lying onher back too. If User A's lying on his or her back, because his or herback hurts and he or she's writing up above them in the sky, one optionis that the virtual world appears to User A so that User B is also lyingnext to him doing the exact same thing. But for User B standing up orsitting, User A looks like he or she's standing up or sittingrespectively, as each user's unique perspective determines thepositioning of the other avatars in the room.

Avatar Interactions in VR

Referring to FIG. 27, a user's avatar 275 is experienced by the user asif it is their own body, in real-life, in real-time. That is, they cansee the parts of their avatar's body that they would normally see inreal life (i.e., they can see their lower body, and can put their handsin front of their face, but they can't see their face or their neck, ortheir whole body, meaning there is no out-of-body experience). There arealso situations where a user might watch a previously recorded VRexperience again, in which case they will observe the recorded sessionas if they were a third person in the room, watching their previousavatar as if it were an out-of-body experience.

Multiuser VR lends itself to many interactive opportunities betweenusers, which are much more personable and “real” feeling than videocalls. For example, because each user has a physical representation inan avatar, these avatars can interact with each other like two peoplewould. A programmed VR application of the invention can implement thisin a tutoring specific sense:

i. Tutors and students can high-five in VR, using hand tracking, so thatwhen their hands come together in space there is a sense of hapticfeedback, to make the experience like a high-five in real life. Thisvirtual enactment of a physical connection relies on the intersection ofthe virtual avatar interactions on the X-Y-Z planes. This is tracked inreal time for both users to ensure realistic-feeling interactions forboth users. The acceleration and velocities of both users movements isalso tracked and communicated in the gesture—i.e. a high velocityhigh-five will result in a louder clap, and more generated hapticfeedback (e.g. stars flying off the users hands), when compared to alower velocity clap which would generate a quieter audio and visualresponse.

Some, but not all, of the ways this feedback can be received are asfollows:

-   -   The sound of a high-five “clap” when the hands come together    -   Both users see an outline light up around their hands, or their        whole hands light up briefly    -   Both users see sparks fly off their hands around the high-five    -   The hands “lock together” when they are high-fiving, just for a        second, to give stability around the gesture and prevent the        hands from floating through each other    -   Both users' hands enlarge momentarily as they clap together

ii. Another gesture that will cause a similar experience with handtracking is a hand shake, with the key here being that when the twousers lock hands in a handshake, their hands don't float through eachother, but instead each user's hand is experienced like a solid objectin VR. This means that when two users bring their hands together in VR,the same feedback as above for the high-five can occur, and the handswill remain locked together until one user draws their hand away, whichwill also be recognized by hand tracking.

Other Users Become Transparent to Avoid Obstructed Views

Even in the physical world, an instructor sends 10 kids up to a board,they can't all stand where they want to stand in front of the boardbecause they take up too much physical space and block views; which iswhy teachers have one student at a time come up at a time. But also evenif a teacher puts five students shoulder-to-shoulder in front of awhiteboard, even then, if a teacher is in the classroom, looking at thestudents, the teacher sees the back of the heads of the students; theteacher does not see the eyes of where the student is actually looking;so they can't tell if the student is looking off to the side, or lookingconfused, or copying their friends work. In VR, A programmed VRapplication of the invention can know exactly where each user is lookingbased off of the head tracking that is built into each headset, so ifone user 1 is at the whiteboard, in front of user 2, then user 1 becomestransparent from the perspective of user 2, so that user 2 can see theboard behind (or through) user 1. This transparent representation ofuser 1 might display in VR as a little border of their avatar body sothat user 2 can still see where user 1 is (or where their avatar is) inspace, but user 1 is mostly transparent so that user 2 can see the boardcompletely unobstructed through the body of user 1. Then, when user 2turns their head farther, such that user 1 is longer blocking theirview, then user 1's body, (or their avatar) automatically comes back. Soonly when one user is obstructing the other user's view of the board, dothey actually get transparent, and this is triggered automatically.Alternatively, users may also choose to turn this option off insettings.

The default here is that an outline remains of a transparent user A(5A), as this allows user B (5B) to still see where the users arepointing when they are pointing in front of the board segment 15 asshown in FIG. 28.

Win embodiments, a programmed VR application of the invention is able todetermine the coordinates of everything in the VR space (X-Y-Zcoordinates), from all users' perspectives at all times. This coordinateinformation of where each user appears to be, in conjunction with thecoordinate information of the orientation of the board in space, meansthat the VR application of the invention can determine when a user isblocking the board. This feature can also be applied when users areblocking content in the open space in VR too. For example, if User 2 isdrawing in 3D in space (not necessarily on information board), and User1 walks between User 2 and his or her drawing (not necessarily oninformation board), then User 1 would automatically become transparenttoo.

Web-User Avatar Controls Via Mouse, Keys and Webcam

Another way that users can access a VR application of the invention isvia a web browser or desktop application and not a head-set device. Thismeans that they can appear to other users as an avatar as if they werealso in VR, but they are actually accessing the application throughtheir web browser or a desktop application, instead of VR. The useraccessing through the web browser or desktop application would still seethe board and all the other users in the room. The user accessing theapplication from their web browser is able to use their mouse or theirkeys on their desktop or laptop computer, or even use their webcam onthe user's computer, to control the user's avatar in VR, so that theuser's avatar movement mimics the movement that the user is generatingfrom the user's mouse or keypad or actual face. For example usingdirectional keys to make an avatar walk through the virtual room, orusing my mouse to turn the direction that my avatar is facing.

If a user is using a webcam on a computer, instead of showing a video ofthe user brought into the VR world, other users are seeing that webcamuser's face translated into an emoji or an avatar, which will be movedby that user's facial expression as seen in the webcam. This is knowntechnology and can be utilized it in the context of tutoring in a VRenvironment of the present invention.

So whether the user logs in through a virtual reality headset, orwhether the user logs in through a website, either way, a user can havean avatar that exists for the user and for the other users of thevirtual reality environment in the same VR room, and the user treatedequally whether entering from a VR headset or web browser or desktopapplication.

Tutor Controls

There are benefits to the tutor having control over some settings in amulti-user scenario. Whoever is the host (or tutor) of the room, is ableto hide tools on the palette of the other person; to disable tools ofthe other person that they won't need for that session. For example, ifa math tutorial is in process, then the chemistry tools can be hidden asthey won't be needed.

There are several ways which the tutor control setting can be allocatedinside the application, which can be toggled on/off inside the userpalette.

First to room: Whoever gets to room first is set by default to be theroom “host”. This is typically the tutor, however the host rights can behanded to another user if necessary. If a student has created their ownroom, then they will be the default host of the room. This is helpfulbecause they will have admin control of the room when they are studyingwithout the tutor. However, when they invite their tutor to the room,the tutor automatically is given admin control of the room for theduration of their time in the room. When the tutor leaves, the admincontrol automatically is reassigned to the student.

Based on hierarchy: the users have their positions allocated (tutor orstudent) prior to a session. Therefore the tutor is given admin/hostrights, regardless of who enters into the room first.

Switchable mid-session: The host authority is able to be handed to adifferent user mid-session. This would be helpful if the tutor needed togive the student authority to make changes for some reason, or ifsomeone outside of the usual tutor-student relationship was added to theroom who needed to have admin control settings.

Temporary control: The tutor (or existing host) is able to hand overtemporary control to another user in the admin panel of the userpalette. This changes the host control to another user for a pre-setperiod of time, before defaulting back to the original host. It can alsobe overridden by the original host at any time, who can take backcontrol of the room if they desire.

Multiple hosts: Multiple hosts can be established by the tutor ororiginal room host, which would be helpful if the student wanted toremain host of the room that they have created to study in over time,but they wanted to give dual control to the tutor for the duration oftheir tutoring session together.

Information/Content Management in VR Ecosystem

Multiple Ways to Input Files & Data into the VR Application

There are many ways that users can bring data and files into the VRapplication in an embodiment of the invention. These files may be fromall different types of image or document file types, and they can bebrought in by any user to be used alone in their VR whiteboard room, orthey can be collaborated on with other users. That is, any file that Iupload onto the virtual reality whiteboard in a multiuser room will alsobe visible and available to any other user in that room. The files canbe drawn on top of and saved or exported again to be used in thereal-world, or they can be saved in the VR application in an embodimentof the invention for users to come back to at a later date. There aremany ways that users can bring files into the room, including but notlimited to textbook import from OpenStax integration, Tutoring companyVideos, text in from the user's cell phone, upload from externalwebsites, Google Drive and Dropbox integrations, or accessing the user'slocal drive through hardware. In addition to these approaches is a novelidea as explained below:

File Import Via Oculus Cameras

Use Oculus cameras to take photos of what the user is working on to addit to the user's virtual room. Background information: The Oculus is thecompany that makes VR hardware. The Quest is one of the devices theyhave right now, the Rift is another one. Oculus Quest has a featurecalled Pass Through where the user can tap the user's headset twice withthe user's finger, or the user can click a button inside of the virtualreality game that the user is playing and it will pause the game theuser are in and activate the external device cameras, so that the usercan see what's around the user. This feature is often set to activateautomatically when a user gets too close to the edge of their designatedplaying area, to prevent injuries and tripping hazards. Many differentVR devices have external cameras on the front, and they are becomingincreasingly common. These external cameras are also now used for handtracking, so that users can use their hands instead of having to holdcontrollers, due to the external cameras tracking the motion of theusers hands and translating this into VR.

Our application uses the front facing cameras to enable users to take aphoto of the material they are wanting to bring into the virtual room,and upload it directly into the room or into their user palette in-app,or straight onto the virtual whiteboard. Each of those are differentiterations and are a setting which can be selected by the user. Thefollowing is an example of how this would be used: a student (User B,for example) based in LA is accessing the VR tutoring room from theirhome through the Oculus Quest 2. They are meeting with their tutor, UserA, who is based in Florida. User A says to User B, “Hey, what have theuser been working on?” And she says, “Well, I've got some homework.” Andhe or she says, “Okay, well let me see it.” She says, “Okay.” User B canthen press a button with her finger or the controller, or with a gestureif she's using hand tracking, which chooses the option from her toolmenu. This option is “Take a photo” for file import, and this option isshown to User B in virtual reality, through the UI tool menu. Whathappens next is that the VR headset's external facing cameras turn onand User B sees what's in her surroundings using the cameras with theheadset, while the headset is still over her eyes. This shows User B aview of the world around her, so she can see her bedroom, and her desk,or wherever she is, and the device shows the room with a “frame” or a“box” to guide and demonstrate to User B the dimensions or the area ofthe photo of the file she is about to bring in. This is just like theguide box the user get when posting banking cheques, how the guide linesshow the user whether or not the cheque is in frame. User B is able tosee her homework on her desk, through the external cameras, line it upwithin the augmented reality frame. She can move her head up and downand side to side to make sure that the homework is correctly in theframe. User B can then take a photo using a controller button or handgesture, or in another embodiment this could take automatically whenfocused on an image or a file for two sections; the headset thencaptures this content. As soon as the photo has been taken, it is shownto User B, with a dialogue box asking “Is this a good photo?” User Bselects, “Yes.” which then saves the photo to User B's device, such thatit can be brought into VR through her virtual tool kit, in-app in VR. Aprogrammed VR application of the invention is using augmented realityhere in this way, by showing a virtual frame on top of the physicalworld as seen through the external cameras on the device. (Note: Virtualreality meaning that the user is in a closed ecosystem of just being adifferent world. Augmented reality means it layers virtual components ontop of the user's current reality, the actual real-world.). A programmedVR application of the invention is using augmented reality as a featurewithin our virtual reality app to get educational content or any contentusing cameras on a device, into virtual reality.

Tutoring Company Videos Generally and Writing on Top of Videos

A unique feature for videos, available both on Tutoring company videos,but also for Youtube and other videos available online, is that the useris able to write on the video as it plays, and the writing is stored asit is overlaid on the videos in real time so that a user can play thevideo back and their writing will remain overlaid on it at the rightmoment.

This means that a user can bring in a video from Youtube or Tutoringcompany or anywhere, into VR, as a two dimensional video. It's in 2D butthen the user can draw on top of it, as it is being played on thewhiteboard, which is interactive in VR. The user would simply draw onthe board as they usually would, in VR, with hand tracking and theirvirtual tool kit, and the drawing would be overlaid on the video for thenext time the video was watched. For example, in one scenario considerthat five minutes into the video, the user chose to draw on the video.Well, when the user gets to five minutes into the video, the next timethey watch the video, the writing automatically starts being re-drawn ontop of the video exactly as the user had done it originally. And thenwherever the user move that video around (i.e. on the whiteboard, as theuser can move the video screen to different placement on the whiteboard)the drawing sticks to it. The drawing (as a vector) is just stuck to theframes of the video, for all subsequent watches of the video.

How Tutors can make Money when not in the VR Application

Background information: The problem with private tutoring as a businessmodel is that income potential for the tutor is intrinsically tied tothe time that the tutor can spend in private tutoring sessions. Forexample, if the user make $50 an hour, and in one scenario consider theuser can work 40 hours a week, then there are 2000 hours in a year whichthe user can spend tutoring, and the user is charging $50 an hour, sothe user makes $100,000 a year. The problem is there's no way to makethat increase without working more chargeable hours, because time isfinite. So what the user have to do is that the user have to create moretime. The way to create more time is to have more students, but notnecessarily groups, which reduces the effectiveness and personal feel ofa lesson, and also reduces the charge out rate the user are able tocharge.

There are several different ways that tutors can use our virtual realityapp to make more income by leveraging their time. Some of these examplesare listed below:

Pre-Set Classrooms/Stations

Tutors can leverage the time by creating preset classrooms, which theycan create prior to going into a tutoring session with the student, orprovide them to a student asynchronously, and not actually attend thesession at all. Using our application, tutors are able to create thesepre-set classrooms or stations in the VR app, either through the webbrowser access or in virtual reality, whichever they prefer. They canset up the room or the board with the content for a full lesson for thestudent. So if the tutor is teaching chemistry, for example, they startwith the textbook chapter for that day's lesson, and they write up asummary of the key pieces of the theory. Then they put in someinteractive flashcards for the student to revise the theory. They walkthrough a worked example of a practice question, where the student canhit play and watch the tutor actually work through this example invirtual reality, as if in real time. On the next board section, there isan empty worked example, which has gaps for the student to fill inanswers as they go. And this also includes some kind of artificialintelligence or even just a simple preset way for the board to let thestudent know immediately that they have got the answer to questionright, particularly for simple answers, like numerical answers in math,or an economics questions where the user are just sliding equilibriumpoints around on a graph. The board automatically tells the student ifthey'd got this correct. Essentially the tutors set up these presetrooms and make them available to students to access without them throughan asynchronous method. The tutor provides these lessons in a referencelibrary of different lessons. So a student goes through and selects froma list that they wanted to study “chapter five chemistry” and go throughand do an hour long preset lesson where the tutor isn't required; fullyasynchronously.

There are different ways that the student gets assistance from the tutorin embodiments of the invention:

i. One is that the student is able to get asynchronous assistance. Thestudent can leave notes for the tutor and a particular color on theboard so that the tutor could come back later, answer the questions andleave feedback for the student again, to come back to.

ii. The second option requires the tutor being available, or “on-call”to be able to assist the students when they need it. The student, invirtual reality, could raise their hand and if the tutor is available,the tutor will get a notification that the student has a question thatthey can then answer for that student. The tutor can then quickly seewhere the student is up to in the lesson by viewing the room recordingthrough webview, or in VR, depending on whether they are on theircomputer or wearing their headset when the student raises their hand,and can then answer the student's question as an avatar in-app. Thismeans that the student gets the experience that the tutor is workingindependently with them, but in reality, the tutor is supervisingmultiple students at once.

iii. In addition to that, another option is that once a student hasasked a question, the next student who goes through that sameasynchronous class also has access to the question and the tutor'sresponse as recorded by the tutor. That is, if one student has alreadybeen through the room and asked a question, then the next student whogoes through the pre-set class will see an icon on the point in theboard where the question was asked, and they can choose to watch thequestion & the tutor working through that question with the student ifthey want to, or they can skip it and carry on if they don't have thatsame question. So as they are going through the board and they recognizethey have the same question, they can watch the question being answeredby the tutor, which prevents the tutor from having to record that sameresponse to the same question again.

This means that any tutor is able to create a library of 10 or 20 or 100preset rooms that they've set up that are available for students topurchase and to go and work through asynchronously in virtual reality.And the tutor can essentially be earning income off all 100 preset roomswhere there may be an infinite number of students in each of these atany time. These rooms can also be used for synchronous tutoring, by thetutor simply meeting the student in the room.

By collecting feedback from users a programmed VR application of theinvention then can use this feedback in combination with AI to generatevalue-add, such as providing users with popular or trending lessons orfrequently asked questions (within the lesson), or even the matching ofa teacher's classroom to a user's learning style/pace/ability. Forexample, a teacher may have different versions of the same course thatapply to different students (e.g. one lesson may be more hands on/3Dinteractive for some students vs a more lecture-based approach for otherstudents). Tailoring a teaching or tutoring style can continue to evolveand improve over time as the VR application obtains more feedback fromusers and the AI improves over time.

Students able to review and record certain parts of a VR lesson:

Students might want to review and record just particular parts oflessons, either the asynchronous lessons or actual real time lessonswith a tutor. Within the virtual reality application programmed inembodiments in the invention, potentially on the user's tool palette orthrough the user settings, a user is able to hit record and take a videorecording of exactly what is happening in the room and on the board atthat time. There are several ways that hitting record works:

If one is immersed in a lesson and after 10 seconds the student believesthe current segment to be valuable, there is a one click option torecord the past 10 or 30 seconds (or 2, 5 or 10 minutes or otherpreferable time increment) which prevents the users from having toscroll back to the starting point of that section and then indicate theending point to save.

There is an intelligent breakdown of the lesson that can automaticallysegment aspects where it is a one click save—the student can hit“record” at any time, and the system will automatically save that entire“segment” of the lesson plan.

Each board segment can be captured as a recording, as a tutor is likelyto use each board segment as a section of the lesson, so the user coulduse their user palette to set the recording from a particular boardsegment which they can see the preview of on their boards.

Natural language processing may be used if the user knows what topicthey want to record the segment of, A user can use natural languageprocessing to say “record from the first time “hydrogen” is mentioned”and the recording function is triggered when an instructor says“hydrogen.”

Another option is if a user wants a complete clip, is that the user canhit rewind and go back in time to the exact point where the user wantsto begin the recording. It will be appreciated that the virtualinformation sharing environment is being recorded in such instances sothat the user can control a rewind function to initiate a recording at aprior moment in time.

In embodiments, a user's recording is stored on the board as a bookmarkor a quick clip that the student can then go back and hit play andrelive that section of the lesson or that section of the board whenthey're reviewing it in their own time later on. There are at least twoembodiments for how this functionality is implemented:

i. Referring to FIG. 29, the student sees the recording like a videorecording that sits on the board. It would be 2D video like conventionalvideo clips and be focused on the tutor and the board.

ii. With reference to FIG. 30, a student can alternatively choose to“step through” into the video and relive it. This would bring thatrecording in 3D into virtual reality around them and relive it as anonlooking additional user in the room, in addition to their tutor'savatar and also their own avatar from the time of the recording.

Another optional feature is forcing the pause button in the VRenvironment application in an embodiment of the invention. A tutor canchoose to set-up a particular pre-recorded lesson. The pause feature isenabled at a certain point in a pre-set room, and automatically pausesthe lesson at particular point in the lesson and then the user has toactively unpause to continue watching. Another option, is that insteadof forcing the pause to turn on, the system instead prompts a userasking if the user would like to pause? This enables the user to take amicro-break to ensure that they have learnt the applicable content, andcan also be accompanied by a mini-quiz question that the tutor canchoose to set.

User Controls

In embodiments of the invention, when reviewing recorded content,students have the ability to control the speed that things aremoving—both the things they see on the board and also thetutoring/teaching speed. Users also have the ability topause/freeze/stop the instruction. In addition, users can also choose tomake the avatars disappear so they have an unobstructed view of theboard, or they could keep the avatar's head but make the hands disappearso they don't block the board content.

Some other customizations for recorded lessons are listed below:

-   -   Users are able to ‘save’ notes in a file to print, or if        ‘teaching’ on a virtual board, then save those boards as well.    -   Users are able to save the video for future review, either in VR        or in a web-browser.    -   Ability to pick up where users left off or reference        items/topics gone over in previous sessions in a later session.    -   Ability to have a reference function built in so if for some        reason the student needs to look something up, they can do that        in the VR setting.    -   Ability to write formulas/text/math in space as an object and        have objects be able to be manipulated in the space around the        user. For example, a user may move the drawing of a normal        vector on a surface the user has drawn.    -   Tutor can mark certain things that change for others/students        and mark other things that do not change.

Time Warp and Multi-User Permissions

In certain embodiments:

i. A VR server is always recording a VR session/conference betweenmultiple users, but the users may not be aware.

ii. During the session, if User A says, “darn, I wish I had beenrecording this!” He or she just clicks the ‘record’ button and the videoserver prompts with “do the user want to record going forward or do theuser also want some recording from before right this second?” If he orshe picks the latter, it asks him to drag a slider on a timeline bar,back to where he or she wants the video to start.

iii. At this point, up pops up a message to all the people in the VRconference. It says “here is a video user and a slider, go skim throughthe recording selected by User A; does User A have the user's permissionto save what happened in the past?’.

iv. If everyone clicks ‘yes’, then the recording from the past is savedin the cloud for User A or to User A's hard drive.

v. If anyone says ‘no’, then either (1) the recording is not saved atall or (2) the recording is saved, But the avatar and voice of theperson who said ‘no’ is not saved. So the user have a recording of agroup of people but the user just don't hear the one person or see them.

vi. Note that if someone has left already, the user must get theirpermission to show their audio/avatar video stream. So, an e-mail issent to their e-mail address on file (or their VR account) to askpermission, and everything is on hold as far as saving that recording ofthat one person until that one person clicks ‘yes’ in their e-mail. Therecording is available to everyone, But the person who left early justisn't shown in the video (or audio).

vii. That is, If that one person who left earlier doesn't click yes orhasn't clicked yes on the permission e-mail, a recording is still saved.The recording simply doesn't show the audio or avatar video of theperson who hasn't approved the video.

viii. If and when the person does approve it, the recording is changedto be a nice full recording (that is, unhide the audio and video of theperson that hadn't granted permission previously . . . but now has).

ix. Again, if someone leaves early, their audio/avatar video will behidden. But realize that that person left early, so there is ALSO somerecent time where that person wasn't on the VR conference. Andtherefore, the user do get some full recording (with everyone). Forexample, in one scenario consider it's 20 minute into the meeting, andperson Z left at the 15 minute mark. Right now (20 minute mark) the userdecides the user should have recorded some of the meeting . . . the userwants to record from the 10 minute mark until now (the 20 min mark). Ifthe person Z (who left at the 15 min mark) has not yet said ‘permissiongranted’, then the user have a missing-one-person-recording for minutemarker 10 to minute market 15, and then from minute marker 15 to minutemarker 20 the user have a ‘full recording’ (since person Z had alreadyleft at the 15 minute mark).

3. Email/Notification Screen with Audio & Hand Tracking

The board displays all the user's emails or notifications as a grid oftiles in front of the user (on the board, or curving around the user ina semicircle), with an image of the sender as the title, and their nameand the subject line of the email overlaid over the tile. How this worksis outlined below:

-   -   Each tile also has a number on it, left to right for example        1-12 so the user can use audio to SAY the number and that will        bring up the email of that number in front of the user, bigger        than the others.    -   The user can then say close, or delete, or reply, or reply all,        and control the email with the user's voice. The user could even        say “reply: Hi User A, thank the user for the user's message, I        will follow up, Cheers, Portia” and then say “Send” and it will        send.    -   Over time a programmed VR application of the invention can apply        AI to make important emails show as bigger emails, or the user        can arrange the tiles into columns for importance (i.e. most        important notifications on the left, and then less important        moving right)    -   The user may also use hand tracking to point and select or        click.    -   The user may use the user's hands to swipe through        notifications; swipe left to delete, swipe right to reply    -   News ticker: people messaging the user while the user is working        show up at the bottom. Or news events or the weather.    -   For tutors, when they get a new booking or a TMS notification,        they can see it here while in VR.

Interfaces for email services may be done in several different ways:

-   -   One option is that the user can sign in through a web-connected        VR headset, straight into the user's mail integration    -   Device has an email client installed on the device which        synchronizes between the headset and the desktop    -   Pop3 which links the user's VR device to the user's laptop    -   If the user's email client is installed on the user's headset,        the user use IMAP server to sync in real time all the time    -   The user's computer serves the information through the user's        headset, so the user's headset becomes an external monitor,        mouse and external keyboard

Syncing Data in Low Bandwidth Situations

During 1:1 tutoring, the only thing that gets streamed is the locationof the new writing on the whiteboard. The tutee's local app on theirheadset already has all the marker graphics and everything else, so theonly new content is when a programmed VR application of the inventionwill have to send the coordinates of the marker/pen and 1 or 0 torepresent whether the pen is writing or not. The local device stores allof the coordinate information that comes in, so it is adding to thewhiteboard at all times. All of the data is stored in the flash memoryon the device—can be loaded via SD card, USB, or downloaded from theinternet.

Pre-Loading Boards to Reduce Buffering:

As users navigate through the boards together, or when a user navigatesalone, then the boards around them are preloaded.

E.G. If user one is on board 10 and user two is on board 50, then aprogrammed VR application of the invention would pre-load boards 9,10,11and also boards 49,50,51 for both users. A programmed VR application ofthe invention can pre-load these boards for both users because eventhough the users are in one place, they are likely to teleport betweeneach other's points too.

The next priority for preloading boards to reduce buffering would be theadditional board segments between the two users, as they are likely tocontinue navigating together in this way.

Real-Time Closed Captions

A programmed VR application of the invention could have real-time closedcaptions happening, so while the teacher is speaking, the captions arebelow them. This is possible via Google Pixel Bugs, Microsoft AzureSpeech Translation, AWS Elastic Transcoder, etc. All are available viaplug-and-play API, and offer AI-based learning.

-   -   it may be for ALL students to see, or a programmed VR        application of the invention can let each student turn off and        on the captions.    -   the captions may be in different languages for each kid.    -   the teacher would not want to see their own captions. That is,        whoever is speaking, the captions are not showing to that person        but are showing to everyone else that has captions turned on.    -   so the person may have ‘show me captions’ toggled ON, but the        system is smart and knows that even if the user have that on,        the user do not want to see the user's own captions.    -   the captions may show up right under the avatar of who is        talking, or may show up under the board, so when the user are        looking at the board, the user see the captions under the board        as the user hear them, or, may be at bottom of the user's        screen, so as the user look at the teacher or move around to        look at the board, the captions stay at the bottom of the user's        view, no matter where all the user decide to look around

In terms of how this process occurs, the following takes place:

-   -   The audio is captured by the microphone in the users device in        real time, as they are speaking    -   This audio is immediately transcribed using a natural language        processing engine that already exists    -   This transcription is transferred over the internet to the other        uses headset, in text format    -   This text is displayed in the other user's visual field in their        headset

In one exemplary embodiment in which a hearing impaired user isparticipating in VR environment with users that are not hearingimpaired, the hearing impaired user can sees words on the torsos oftalking users. However, users that are not hearing impaired preferablysee the text captions on torsos since they would not have toggled thisfeature on, but the impaired person would have toggled this feature on.However, in some embodiments, any user in the VR environment (whether ornot hearing impaired) could turn on the display of text captions on oneor more other users' torsos if, for example, they find another user isspeaking to softly or is hard to understand. It will be appreciated thatit other embodiments the text captioning might appear in other locationsnear the user represented in the VR environment and not just on thetorso, such as above or below a user, in a speaking box near the user orin other like locations that allow other users to “see” that thespeaking user is saying. Each user may have control settings to turn“off” or “on” which users have text captions and/or where the captionsappear for that speaking user.

Real-Time Sign-Language Translation

Through natural language processing integrations, a programmed VRapplication of the invention is are able to generate real time signlanguage translations which would be delivered via an avatar that isnext to the user in VR. For example, in a pre-recorded module, a usercould turn on real-time sign language in their user palette which wouldmake the avatar delivering the pre-recorded content begin alsotranslating the audio into sign language.

In real-time interactions, immediate language processing may also beused to generate avatar sign language as two people interact with eachother in VR in real time. If someone was hearing impaired, they coulduse hand tracking on their device to sign to the other user, who couldrespond in audio which would be translated into sign language in realtime.

Handwriting Recognition

Executing in the VR application is handwriting recognition, where twothings occur:

The user's handwriting on the board is corrected into text as theywrite. So that if the user write with messy writing, this is smoothedinto nice, readable text for the user's students or the other people inthe room with the user. This may be a setting turned off or on in theuser palette, or it may be a special pen in the VR app whichautomatically turns handwriting into text as the user write.

All writing is transformed to text with handwriting recognition, and isall held in the server, whether the user has this setting turned on ornot. That is, even if the user has their regular handwriting on theboard, a programmed VR application of the invention is are runninghandwriting recognition in the background ONLY, for search capabilities.This means the user may search “physics” in their user palette, andevery time they have written “physics”, even if they just wrote it withtheir own handwriting, the board will be able to highlight these areasfor them.

That is, a programmed VR application of the invention is decoupling thefunctionality of the handwriting recognition for search capabilitieswith actually fixing the handwriting for someone on the board. Thesearch functionality always occurs in the background, but the user hasthe option to turn off or on the fixing on the board, the user facingside of this feature.

From the users' perspective, they can select if they want handwritingrecognition on or off, but really, that is not what they are selecting,a programmed VR application of the invention has it ON all the time, nomatter what the user tells us. The user thinks they are turning it offbut really they are just turning off the functionality of turning theirwriting into text. A programmed VR application of the invention is stilltranslating and keeping on the recognition, no matter what, so it can beused for the following functions:

i. Tagging sessions by subject/content automatically

ii. Letting the user search and find things in their own room

Input/Sensors/User Tracking & Monitoring

Multiuser Feedback from Users

Background information: Quite unique to tutoring, when compared to othermultiuser VR games or applications, is that it is very important for thetutor to understand how the student is getting on during the lesson; howmuch they are understanding, if they are confused or bored or distractedor stressed out. This is something that was discussed by many teachersas a key difficulty with moving online for teaching; they couldn't readthe body language of the students, or watch their eyes to see what theywere looking at. One way to deal with this is that the tutor has to askthe students repeatedly, “Does this make sense?”; but students oftenwon't tell tutors that they are struggling anyway. There are severalmethods for collecting user feedback in virtual reality that aprogrammed VR application of the invention captures and make availableto tutors while they are tutoring in the VR application in real time,but also for them to see retrospectively to reflect on the lesson andplan for their next lessons.

Head Movements and Eye Tracking for Multiuser Feedback

A programmed VR application of the invention detects head movementsthrough the VR headset that the users are both wearing. Some of the keyhead movements that will be useful for this feedback are:

-   -   The user is nodding their head ever so slightly, showing that        they are starting to understand

Unless they have some neurological disorder (tremor, etc.), however itis possible to distinguish between them with an AI application.

-   -   If a student is often looking back to the left or to the West,        which is in the direction of the previous material, then this        shows that they are confused or not keeping up.

The tutor cannot be watching for these movements in real time while alsotutoring, especially if they have several students at once, so instead aprogrammed VR application of the invention can aggregate all of thisinformation and a programmed VR application of the invention can let thetutor know that this student has been spending too high of a percentageof their time looking back.

There are several ways that this information can be presented to thetutor:

-   -   A virtual reality marker above the student's head, which could        be a slowdown sign    -   A traffic light system either beside or above the students        avatar, or even on their t-shirt, which shows a change in the        color from green (when they understand), to yellow (when they        are starting to look back quite regularly), to red (when they        are looking back very frequently, or are completely distracted        and looking the other way)    -   On the tutors tools palette, or their in-app device, they get a        little notification or a feedback tracker across the bottom of        the screen, which shows them the status of each of their        students and how they're responding.

There will be default thresholds for head movements which correlate tolevels of understanding, which the tutor can change depending on howclosely they want their students to be following. Some examples of howthis threshold may be set are as follows:

-   -   Number of glances back to the left per minute: for example, 3-5        glances to the left in 1 minute=yellow, 5 or more glances to the        left in a minute=red.    -   Percentage of time: for example more than 20% of the time not        looking at the tutor=yellow, more than 40% of the time not        looking at the tutor=red.

A similar feedback system may also be applied for eye tracking, where aprogrammed VR application of the invention can capture the movement of auser's eyes in their headset, and aggregate this feedback into emotionalcues that a programmed VR application of the invention can deliver tothe tutor. Some of these cues could be:

-   -   The student's eyes are shrinking or squinting ever so slightly,        to show that they are focusing intently, or that they may be        confused;    -   Eyes are moving back and forth a little bit, showing that they        don't understand    -   Eyes are drifting away from the board or aren't focused on the        tutor or the work, showing they are bored, distracted or        disheartened.

This information is presented to tutors in a similar way as described inpreviously, but it may also be displayed in unique ways to show wherestudent's eyes are resting too. Some examples include:

i. Lasers, or non-distracting lines, such as gray dashed lines orsimilar, coming from the eyes of the avatar of the student (which atutor could choose to turn on or off) that show whenever a studentstarts looking back at old material. The tutor can follow the line ofsight of the student to the board to see what they are looking at.

ii. Another embodiment of this is that the tutor could see where thestudent is in space: for example if the student teleported back to anearlier board, and the tutor is teaching multiple students at one time,so they can't keep track of where all the students are at once. Thislights up a little gray number in the tutors admin panel showing whichstudents are in front of which board segments.

iii. Another embodiment is that the user's line of sight shows on theboard like an outlined circle, or a highlighted circle around thecontent that the student is looking at. So instead of a laser coming outof the eye, the area that they are reading from is slightly brighter onthe board so that the tutor can see if further down the board isbrighter, that that's where the student is looking. And that can becolor coded, so there's a circle around the student and a circle aroundthe board segment. That they are looking up, it's both blue, or red, orpurple, and each student would be given a different color, etc.

iv. Adding onto that, there is also a setting that the tutor turns on oroff, where if a tutor pointed at a student with two fingers or threefingers or some kind of gesture, it could turn on the ability for thetutor to see where everybody's eyes are looking or just where their headis facing. Particularly beneficial for a tutor who is more advanced, whois likely to want a little gray, very faint, gray ray of light,connecting each avatar to exactly where they're looking on the boardsegment, or on previous board segments.

v. In Google docs, for example, the user can see where everyone else inthe document is based on their color coded cursor icon; or how the usercan see the color-coded outline when people are hovering in a cell inGoogle sheets. This same system may be used in combination with the heador eye tracking system: The tutor sees a color coded outline around theboards that each student is standing in front of or reading from, with apink, a red, blue, different circles for each person, and a little namefloating with the circle to show exactly what they're looking at. So theuser say, “Okay guys, I put 10 problems on one board segment. I want theuser to walk up to the one that looks easiest to the user and startworking on it.” And the user would have little circles on what exactlythey're looking at.

A key in the embodiment is that the students would not see this, but thetutor could turn this on for the tutor to be able to see where all thestudents are looking.

EEG or fNIR, fMRI, PET, MEG, TMS, CT Scan, Eye-Tracking Sensors for UserEngagement and Cognitive Load Feedback (using Neuralink technology):

One of the key ideas in the education field is that of the “GoldilocksZone” of brain activity (alpha and theta waves) where the student ischallenged enough that they are learning effectively, but not so muchthat they are confused. Students learn most effectively when they areengaged, but not when their brain gets too overloaded; if cognitive loadis too high so they won't learn anything, but it also can't be too lowbecause then the user is not engaged. The research is so incomplete andnew that at a minimum, the best guess currently is that everybody has adifferent Goldilocks zone.

In one embodiment, EEF or fNIR sensors are worn with the VR headset tomeasure brain activity and oxygen levels in the user's brain. Thisinformation is then broadcast from the students to the tutor, notbetween the students, but just to the tutor. A programmed VR applicationof the invention can very quickly build that feedback loop. The systemconstantly keeps track of each student's engagement and different brainactivities, alpha waves and theta waves being the big ones, and alsoaggregating this information with eye tracking and head trackinginformation too. For example, when students are confused, the momentbefore they look back means they're probably confused, so a programmedVR application of the invention can track this head movement and eyemovement and the associated brain waves. This provides real timefeedback for the tutor on each student's brain activity and alsoengagement, and connections between eye/head movement and brain activitycould be drawn, to assist the tutor in keeping each student highlyengaged and recognizing their confusion or distraction before it kickedin.

Another feature that a programmed VR application of the invention canincorporate is verbal cues from students too, for example when thestudent says: “Oh, I got it.” The functionality implements eitherthrough natural language processing or the tutor themselves presses abutton or clicks something, to say, “Okay. Mark that time right therebecause that student got it.” Other information that is capture includesthe tutor marking times when students seem like they were learning, theywere engaged, they seem like they were learning and it led to the “aha”moment because the “oho” moment probably has a whole different brainsignature than the learning part that leads up to it. All of thisinformation is later aggregated with the EEG, fNIR, fMRI and eye/headtracking data to draw a clear picture regarding each individualstudent's learning patterns, preferences & Goldilocks Zone.

Another way that EEG and fNIR data is used, is that sensors are used forautomatic VR settings, for example setting the in-app brightness level,a programmed VR application of the invention can auto measure brightnessthat works for each individual's eyes based on their brain response. Ifthey're feeling pain in the brain and the brain is not happy, then thebrightness is too high.

Engagement measurement display 310 (see FIG. 31) and cognitive loadmeasurement display 320 (see FIG. 32) are the two key metrics that aprogrammed VR application of the invention can share with tutors.Different inputs will provide information for both; but this will becombined to give a simple, user-friendly output for tutors in their userpalette admin panel. A programmed VR application of the invention canpresent the data such that it can be quickly interpreted so they canimprove the tutoring session for the student in real-time. What is newand tutoring specific is that for cognitive load, after a certain “blisspoint” going further higher in load becomes worse for learning, comparedto engagement, where higher measurements are better for learning.Therefore there are several different ways a programmed VR applicationof the invention can communicate this information: 1-5 scale of thecognitive load/engagement. −2, −1, 0 (Goldilocks Zone), 1, 2 (too muchcognitive load)

A traffic light system for both of these metrics. Cognitive load goes inboth directions, as there is a Goldilocks zone, so the barometer will begreen in the middle

Graphs, numbers, diagrams, traffic light systems to make it easy for thetutor to digest and act as a result of, regardless of the input it comesfrom.

In terms of how this is displayed: Either in the user palette or just atthe bottom of the screen at all times, or both (see FIG. 33). As shownin FIG. 33, tutor 335 can view each respective students' (331A, 331B and333C) measurements with respect to the information board 10 on the tutorpalette 330. Measurements for student 331A correspond to engagementmeasurement display 310A and cognitive load measurement display 320A(shown on tutor palette 330). Measurements for student 331B correspondto engagement measurement display 310B and cognitive load measurementdisplay 320B (shown on tutor palette 330). Measurements for student 331Ccorrespond to engagement measurement display 310C and cognitive loadmeasurement display 320C (shown on tutor palette 330). Tutors can alsocustomize settings as to when they are notified about the status of theengagement data.

Some of the technologies a programmed VR application of the inventioncan use to measure brain stimulation, engagement and cognitive load arethe following:

Computerized Tomography (CT)

Magnetic Resonance Imaging (MRI)

Transcranial Magnetic Stimulation (TMS)

Electroencephalography (EEG)

Functional Magnetic Resonance Imaging (fMRI)

Positron Emission Tomography (PET)

Magnetoencephalography (MEG)

Pen/Stylus tracking and hardware:

It is known that Bluetooth pens & styluses are used extensively withother tutoring technologies, but these devices are yet to be broughteffectively into Virtual Reality information sharing environments.Referring to FIG. 34, there are several different embodiments for a user5 to use a Bluetooth pen 340 in connection with a VR headset 342 in a VRenvironment programmed for use with such devices.

VR Compatible Pen Hardware:

Existing Bluetooth pen mouse 350, as shown in FIG. 35, is used forcomputers but not used in current VR environments.

In one embodiment a Bluetooth pen is designed specifically forfunctionality in our VR app, but is also compatible with other VR apps.BLE, accelerometers and LIDAR being the currently available hardware,which are integrated relatively easily.

The question is how many external (stationary) sensors are permissiblefor a room—this will allow for lesser or greater accuracy.

The Bluetooth pen connects to the Oculus Quest headset directly throughBluetooth, and has convenient functionality such as the following:

-   -   Clicker on the end of the pen used to choose different tools        (pen, laser pointer, eraser, 3D pen); in VR the user can watch        the user's pen actually changing to be those other tools, so the        user can see what tool the user have at any time as the user        click through the options.    -   Scrolly-roller on the side to zoom in and out, like a        magnification tool.    -   Secondary click button; another button on the side of the pen,        to use for things like activating the teleporting function when        the user has the pen set to be the laser pointer, or to activate        the select tool when the user has the pen set to any other too    -   In this case, the pen click is JUST for functionality (to use        specific features of a tool), and instead of changing tools, and        instead, to change tools the user ROTATE the end of the pen        (like a pen that the user rotate to get the pen tip to come out        in different colors, but with more rotations for more tools)    -   A slider down the side of the pen which is used to change the        width of the pen/eraser/whatever tool is being used at that time        from bigger to smaller    -   A built-in microphone which can automatically transcribe        anything the user say to notes on the board    -   Able to access settings with the pen for different tools        (essentially the customization section), so potentially a second        rotating section (or potentially rotate the other direction) to        change colors. OR a button to open customizations (like the        color wheel for the pen) which the user can then just select        with the pen in space    -   In addition to all of the above: the functionality is able to be        customized in the VR application in an embodiment of the        invention so different parts of the pen which can be scrolled or        clicked or twisted can be coordinated in the VR application in        an embodiment of the invention to change the color of the ink,        for example, by clicking on the pen or using a scrollbar to zoom        in and out on the board and make the content on the board bigger        or smaller, or activating a laser pointer by using a slider on        the side of the pen so that all of the functionality that the        user could use in the VR application in an embodiment of the        invention can also be built into the pen as a sort of very        specific controller.

The external cameras on the VR headset are able to ‘see’ the pen, or thepen could also be connected through Bluetooth or Wi-Fi. And the Questknows the size of it and shape. There are two transmitters in the pen.Or one in the middle with an orientation signal too. Now, AI and handtracking in Quest can be used to know where hands are. No need to trackthe whole pen if there are two transmitters on the edge tips of the pen.So now the device knows it's not missing the hands. In fact it knowswhere the hands are based on how the pen is being held; so the VR handsare just generated around this pen, so the physical pen is used to aidthe accuracy of hand tracking in VR.

Note: The Bluetooth sensors would only track the hand which is holdingthe Bluetooth pen; which means that the other hand (if not also holdinga Bluetooth tracking pen) will be tracked by the external cameras on theVR headset which already exist and are the primary tool for handtracking at this stage.

Another way that a programmed VR application of the invention can trackthe pen orientation and movement is with a gyroscope. Based on the slopebetween sensors and the orientation from the gyroscope, the user knowexactly how the pen is oriented in three dimensional space. That wouldhelp make the drawing really, really accurate on the board. It wouldalso make it more comfortable for the user because they'd be holding apen, in particular, a pen that's the exact same size as what they'reseeing inside of virtual reality. It looks the same so they would nothave cognitive dissonance.

Another embodiment is to put a special cap, or a special sleeve on anexisting pen. This sleeve or cap would have two sensors to achieve thesame outcome as explained above. This means that people literally usethe exact pen they're used to using, but bring it into virtual reality.

Now the other benefit here is that, again, the user don't have to trackthe hands, or the pen will correct for errors in hand tracking, whichhappen quite frequently. The user don't have to use the front facingcameras, which requires a particular level of lighting to be able towork correctly. If the user have a Bluetooth or Wi-Fi enabled pen, theuser can literally do this in the dark. So if the user is in a darklibrary for some reason, where there's a library where no one has thelights on, the user could still pull up tutoring VR. Anyway, so the penis brought into virtual reality and the user use the user's hand. Itmakes it more comfortable for the user because drawing in virtualreality with nothing in the user's hand in the physical world is verystrange. And then when the user pick up a pen or a marker in thephysical world, but then in virtual reality, it looks different fromwhat the user is holding and the width is different and the weight isdifferent than the user expect, then the user get a sense of cognitivedissonance.

2B. In-App Pen/Stylus Tracking and Matching the Physical World to theVirtual World for Writing Utensils.

When the user bring a pen into VR, the user may change the look and feelof the pen in VR, because it has got an animation on. This means theuser may customize the pen in VR to look exactly the way it looks in thephysical world so that they know, “Oh my God, I just brought a pen intovirtual reality with me. I brought my own physical pen into virtualreality.” Users may customize different settings such as the ink colorand ink width, pen width and length, and they may choose whether theywere using these settings to make their VR pen more realistic, or justto make something fun and obscure like a 2 foot long pink twirlywizard-staff pen.

Another option is that if a user is using a pen from their house, theymay use the cameras on the front of the Oculus and snap a photo of it,and actually generate that pen into VR through the image of the pen.

This includes a process to take a picture, or a set of pictures of anentire structure to make sense of the real (3D object)-E.G. a pen iscylindrical and may have text all the way around the structure, so thisis all be captured in the image when its brought into VR.

This is beneficial to reduce cognitive dissonance, because for example,if the users doesn't have an Expo marker at home, then the VR Expomarker is not going to make a lot of sense if they pick up a pen and theweight of it will feel really good. The user can change the look of theuser's pen or marker based on what the user is holding at the user'shouse in the physical world, either through the cameras or throughunique customization in-app through a library of customizations options,which is another embodiment. Another embodiment is that the user maysearch for the name of the user's pen, and a programmed VR applicationof the invention can have a library of different popular pen optionsready to be selected from. A programmed VR application of the inventioncan have the name of the pen that the user is looking for pre-saved andthen it comes up without having to take a photo and that way aprogrammed VR application of the invention can match the pen.

Physical Tablet with QR Markers Projected into VR

This idea is that of using a physical tablet instead of a virtual one.This tablet has special QR markers placed on the edges or a specialframe with tracked pucks which will help us bring in the physical tabletinto VR as a tracked object. The QR markers will allow the externalcameras on the VR headset to recognize the tablet.

The tablet will be projected into VR as it is. This means that the userwill be able to see a one-one replica of a physical tablet in VR as avirtual tablet. This tablet could be a technical piece of hardware witha user control interface app built into it, or it could be as simple asa legal office pad of paper which has contrasting colored stickers stuckto the corners of it to act as QR markers.

If the technical version of the external tablet/palette, an app on thetablet will help us control various aspects of the VR world includingthe whiteboard in VR. The VR application in an embodiment of theinvention can have students and teachers mode where both project to thewhiteboard. However the teacher can choose what can be projected andwhat not to be projected and even control the scaling ability of thewhiteboard. They are able to design their entire teaching session(lesson plan) on this app entirely in the tablet and bring it into VRand directly project it on the VR whiteboard or if they choose to in aregular physical projector. The VR system simply augments and enhancestheir existing capability. The metaphor is that of an Overhead Projectorwhere the VR whiteboard is the screen where things are projected

Whether technical or just a pad of paper, a programmed VR application ofthe invention is are able to project what is written/drawn on the tabletonto the VR whiteboard, by using the external cameras on the headset andhand tracking recognition paired with the QR markers to recognize thewriting on the pad and translate this to the whiteboard. This aids usersbecause they would not have to strain to hold their arm up to write invirtual space for long periods of time, which would be much morecomfortable for them.

Movement

In virtual reality there are different ways to generate movementdepending on whether the user are using controllers or hand tracking.Our unique embodiments for movement in VR are listed below:

Laser Pointer Triple Functionality

Because tutors often need a laser pointer, and teleporting is one of themost common motions needed in VR, a programmed VR application of theinvention can include a unique, three-part gesture which combines thefunctionality of these features, with the additional feature of a“viewing screen”. The triple functionality gesture, illustrated in FIG.36, is as follows: 1) laser pointer 2) viewing-screen 3) teleporting.

Each user will have one laser for each hand; a green laser 364 in theirright hand, and a red laser 362 in their left hand. These lasers willshoot out at a trajectory from the users index finger, in a straightline to the board, so that the user can intuitively point with theirindex finger and the laser will shoot out. The triple gesturefunctionality is the idea that when the user turn on and off the laserscoming out of the user's fingers, the user can then use those lasers topoint something out to someone, either up close, by shining the laser onthe board, but also for pointing out or identifying something furtheraway, by highlighting another, further away board segment (more than 2boards either side of where the user are standing), and then the usercan use an alternation of the same gesture to teleport to that boardsegment, or to any other point on the floor in the VR room.

The three ways that this tool is likely to be used are outlined below:

i. If the user are shining the lasers on the board in front of the useror either side of it, a programmed VR application of the inventiondetermines that the user are likely aiming to point out content on theboard (As the user are pointing at it with the user's index finger, tobring it to the other user's attention), so the laser pointer isactivated, with a small focus point dot at the end of the laser beam toshow users what the laser-yielding user is pointing at.

ii. If the user are shining the lasers on the board further than oneboard segment either side of the user, then the user won't be able toread the content on the board effectively, so there is little value inthe laser pointer being used at this distance in the way described in(i) above. Therefore, when users shine the laser at boards further thanone board segment either side of them, this highlights the boardsegment, by putting a glowing border around it, to show the user whichboard is selected. An additional setting here which the user can chooseto turn on or off, is that when a distant board segment is highlighted,this then automatically brings up a viewing window, (think of this assimilar to a scope in a shooter video game). When this viewing windowpops up, it basically just brings the board to the user, so that theuser can read the content clearly without having to teleport all the wayto the board, which is useful for a tutor who might be referencing anold piece of work briefly, for example. When this viewing windowappears, there's a left and right button so the user can scroll to boardsegments either side; the user can touch or the user can tap or the usercan shoot and it just kind of cycles through the different boardsegments that are either side of the segment the user had initiallyhighlighted with the laser pointer. If, when looking through the viewingwindow, the user decide the user do actually want to teleport in frontof that board segment, then the user can just hit a button or completethe gesture to activate teleporting. When the user teleports, this isdifferent to sliding down the board as the user would do in thesubsequently described “Jedi Hands” embodiment. When teleporting, theuser moves their position relative to the entire room, which isprimarily the board, but also may include other objects and 3D modules.The user moves as if teleporting to a totally different space in thisroom.

iii. Because there is no content on the ground, users would not use thelaser pointer to point anything out on the ground. Therefore, if theuser is pointing towards the ground, the user is indicating that theuser want to move to a different point on the ground, so the laserautomatically becomes a teleporting arc with a teleporting circle at theend of it, to show the user where they can teleport to. The user canmove this around just by pointing, until they find the spot they want toteleport to, then they just activate the tool to teleport.

As for the gesture itself to activate this, there are several differentgestures that work for this triple functionality, all of which arerecognized with hand tracking through the external cameras in the VRheadset:

i. One way is to use a gun gesture with the user's pointer finger, theuser's index finger pointing out and the user's thumb pointing up. Inthis version, the user turn on and off the laser/teleport tool by“shooting” the gun with the user's thumb trigger by pressing the user'sthumb down to the user's hand; that would toggle on and off the laserpointer. The same gesture, if the lasers are already turned on, is usedto toggle teleporting to the highlighted board segment or teleporting orgoing to the identified teleportation spot.

ii. Another embodiment is to pinch the user's thumb and the user's indexfinger together once or maybe twice, and that toggles the laser featureon or off. When using this embodiment, then pointing the user's laser ata board segment or at the ground automatically loads the viewing window(for when the user is pointing at the board), or the teleporting circle(when pointing at the ground). When these features appear, the userholds their laser pointer steady for 2-seconds to activate the “GO”teleporting button, which will appear about 3 inches in front of theuser's finger. This button will appear after two seconds, and when thebutton is pressed it will immediately teleport the user in front of theassociated board segment, or to the point on the ground that they havespecified.

iii. Another embodiment is that if the user is pointing in any way,shape, or form, even if the laser tool is turned off, if the user pointfor a while at a specific spot on the ground or in a board segment for awhile, a little “GO” button pops up about three inches in front of theuser's finger and the user just move the user's finger forward to pressthe go button or click the user's controller and it brings the userthere.

Jedi-Hands as a Mode of Movement without Teleporting

The idea of the Jedi-hands gesture is to make moving short distances inVR quick and efficient by grabbing the “room” around with the user withtwo hands and using the user's hands to move the room relative to theuser's avatar: therefore moving the user's avatar through space (usedfor teleporting short distances primarily, as it takes a while to move slong distance by this method).

The unique gesture a programmed VR application of the invention may useis that the user first holds their two hands up in front of them. Theylook at the back of their hands, make a first as if to “grab” space inthe room, and then they can pull the room around. As the user move theirclenched hands around, it moves the entire room around, either left andright or forward and backwards. As if the user had grabbed theroom/world around the user, and the user are moving the entireroom/world relative to the user, in proportion to the user's movement.For example, if the user grab in front of the user and pull the user'sclenched fists just slightly towards the user, then the room will movejust slightly towards the user. If the user pull further, the room movescloser. If the user pull the user's clenched fists slightly to the left,then the room moves slightly to the left. When the user release theuser's clenched fists, the room snaps into position where the user havepositioned it. Another embodiment of this is that if the user move theuser's hands away from each other or closer to each other, it changesthe scale or the zoom of the room.

VR Camera Movement

Third option is that the user can take a camcorder in VR and the usercan move wherever the user want, while the user watch. Instead of being“inside” the eyes of someone, the user can instead just move a cameraaround at all times, while the user are watching a video. This means theuser can watch, in 3D, in the user's headset, from anywhere the userwant in the environment that was being recorded. This is analogous tochoosing to be like the skycap in National Football League (NFL) games,or the user can pretend the user are sitting on the ground, looking upat the tutor tutoring the student, or whatever the user wants.

Cheating Prevention with VR

A. Headset movement detection: If the user are taking a test, with a VRheadset on, a programmed VR application of the invention can detectwhether or not that the user have taken the user's headset off, becauseof the gyroscope in the headset. If the user are meant to be taking atest in VR the user will be informed to keep the user's headset on forthe duration of the test. If a student takes their headset off, thismotion will be detected, so a programmed VR application of the inventioncan tell that they have removed their headset to look at their notes orto cheat.

B. Passthrough mode on a VR headsets: Passthrough mode allows users touse the external cameras on their VR headset to see the room around them(this already exists). If a student is taking a test in VR and they turnon passthrough mode, a programmed VR application of the inventiondetermines that they are looking into the real-world, outside of theirheadset. This could be because they are cheating, but it also could bebecause someone has walked into the room to talk to them, for example.Therefore, a programmed VR application of the invention can store all ofthe data that they view while in passthrough mode. This could just betemporary, deleted afterwards for privacy purposes. When a student goesto turn on passthrough, a programmed VR application of the invention cannotify them “The user are in test-mode; we will store the user'spassthrough view data when the user turn this feature on” so that thestudent knows that they are being recorded so they can't cheat.

c. Teacher set environments, tools and resources: the teacher can decidewhich tools the user get and don't get, for the purpose of the test. Forexample, teachers can select from a menu of calculators for the onesthat have the appropriate tools that don't enable cheating. Anotherfeature with calculators is that a programmed VR application of theinvention could clear all previous memory/settings, as many calculatorshave memory in the VR application in an embodiment of the invention. Ifa student is in a teacher-set test environment (which they might accessthrough an access code) then they don't have access to their usual userpalette with their stored personal notes, they instead just have theclean slate test environment as set up by their teacher.

d. Access codes & integrations: For doing the user's homework, (becauseVR is better as It removes all distractions) the user have a code fromClever (see clever.com) or Class Link (see classlink.com) or anotherschool API. The user enters a code which puts the user into “homeworkmode” and puts restrictions on the environment with tools and resourcesthat a student has access to. Teachers set this up for students, withexactly the tools that they need for that week's homework and nothingelse.

e. Audio detection: A programmed VR application of the invention candetect the audio through built-in headset microphone so that if astudent is asking someone for help (for example asking their olderbrother the answer to their test question), then the headset recordsthis. Usually while sitting a test a student shouldn't be using audio atall, so any time speech is detected this audio clip is recorded andstored. This is sent to the teacher in a report, broken down per eachstudent in each class. The teacher can listen to each audio file to seeif the student had cheated. Further, natural language processing canlook for typical speech for cheating; for example “What's the answer to?. . . ” or “Does the user know? . . . ”; these types of speech will berecognized and red flagged immediately for the teacher to review.

f. Eye tracking: Eye tracking can be used to measure a student'sdistraction by tracking where in the headset the students eyes arefocused and for how long, and can also determine if the student haspulled the headset forward to look at something in the real-worldoutside of their platform. If a student is looking down, up, side toside, then this is suspicious as they should be focused inside theplatform; this implies they are likely trying to look outside the edgesof the headset. In this case, external camera data can also be triggered(passthrough feature) and take a snapshot of what the student is lookingat, at that very moment when the eye tracking detects that they werefocused outside of the headset. Further, a programmed VR application ofthe invention is able to blur the users periphery while in the headset,as a programmed VR application of the invention can track where theireyes are looking at any time. A programmed VR application of theinvention can do this by muting/blurring the pixels around the edge oftheir visual field, to ensure that the user is focusing their eyes inthe center of the visual plane at all times. If the majority of thevisual field are blurred, then A programmed VR application of theinvention knows that the user is trying to look into the edges of theirheadset to try and look into the real-world.

g. Light detection: Some headsets detect light coming in through thelenses of the headset, so if light is detected by the headset then aprogrammed VR application of the invention knows that the headset hasbeen pulled down: i.e. the student is looking out of the headset. Thisis recorded too and sent to the teacher in a report.

h. More importantly is that the work inside VR just before and after thestudent pulling their headset down or looking side to side (out theedges of the headset), or looking with passthrough, is recorded. Thismeans that the teacher can deduce that the student was clearly confused(erasing and trying again, looking blankly at the question, attemptingit with incorrect steps) before they do the potential cheating action(removing their headset etc.). If the student is then seen to beconfident in executing the correct working/answer, then the teacher canexpect that the student looked at their real-world notes or cheated insome other way by removing their headset.

i. Students staying in VR but leaving the VR application: If the userleaves the VR application, then no data would be collected, so aprogrammed VR application of the invention can easily track whether astudent has left the VR application to browse the internet, or to usesocial media where they might be cheating.

j. Contact lists: If one student gets the exact same answers as anotherkid; then a programmed VR application of the invention can detect thatthey might have cheated. A programmed VR application of the inventioncan compare student's contact lists to see who they might have cheatedwith. Oculus logs in through Facebook, so a programmed VR application ofthe invention can compare the class list and the student's friend'slist, which gives more information on suspicious cheating activity.

k. Timestamps: When a test or homework session is started and ended, aprogrammed VR application of the invention can record the timestamp ofeach of those actions. Two students are unlikely to be in VR at theexact same time, so timestamps for starting and stopping doing theirhomework would also detect and prevent cheating. If two students arefrequently going in at the exact same time, or immediately one after theother over time, this could indicate cheating.

I. Scratch paper: Currently in the real-world, when students sit onlinetests, teachers often request that students take a photo of theirscratch paper to send in after their test. The issue with this is thatthere is no way to monitor it or to ensure that it is actually thescratch paper that was used by the student; a lot of student's do thisinaccurately or send in fake scratch paper. The teacher would never knowwho's scratch paper it is. In VR, the scratch paper is in the VRapplication, so that the scratch paper can't be faked; the use of thescratch paper is recorded, so the teacher watches the student workingthrough the scratch paper.

m. Students currently use an app called Photo Math (see photomath.com)on their phone, which gives them the full answers to math questions whenthey take a photo of it. Instead, a non-cheating version for in VR,where the user can scan the user's physical homework (from thereal-world) with the user's phone or with passthrough on the user'sheadset to bring it into VR to work on it. There are options here, thatjust the homework can be brought into VR, or a programmed VR applicationof the invention can also have a feature like “request a hint” where thefirst step of the math equation, or a helpful tip about the process ortheory or steps is shared. This can also be disabled completely for testor homework mode if the teacher or the parents prefer.

n. Casting in VR: Teachers or parents are able to disable the castingfeature (which allows people to screen share what they are looking at inVR into a computer or mobile device in the real-world). This wouldprevent students from sharing info from in the VR application in anembodiment of the invention to other students, for example the testquestions as they work through them. This feature is set by the API intoClever or another schooling system, so the teachers could control whenthe casting feature is enabled or disabled. A programmed VR applicationof the invention can also monitor when the casting feature is being usedthrough bandwidth measurements. If the bandwidth is too high then aprogrammed VR application of the invention can determine that thestudent is casting their content to another device, which could implycheating. A programmed VR application of the invention can also thenmonitor what content exactly is being casted, which helps when teachersare uncertain about whether or not cheating is taking place.

o. Biometrics for cheating prevention: to prevent someone else fromusing the user's headset for the user to complete a test.

i. Using eye tracking, a programmed VR application of the invention cantrack the unique eye movements of users. Some HMD have eye trackingbuilt in already (one HMD made by HP is for sale for $600 which includeseye tracking currently).

ii. Using lenses already built into the headset, a programmed VRapplication of the invention can take a screenshot of the user's eyes toprovide an ID shot of the users' eyes. Alternatively, a retina scan iscompleted with this eye tracking/lens, to match the user.

iii. A programmed VR application of the invention can also implementfingerprinting, which requires external hardware, but is added to thedevice with software that prompts the user to scan their fingerprint ata certain point.

iv. Distance between the users hands and their headset at maximumextension, and also at their regular drawing length. Also their dominanthand will be an indication of who is using the headset.

v. Questions (similar to a credit history questionnaire) are asked atthe beginning of the session to identify users.

vi. The easiest method is using a retina scan. This is also the mostsecure (e.g., with fingerprint detection, there are ways to thwart thescan—one user can put on the headset, while another user authenticatesit. This may be defeated by not accepting the fingerprint if anotherbody is nearby.)

Virtual Keyboard

Hand Tracking

If our device knows where the hands are, right, and the user put theuser's hands on the keyboard, and the user and the system says, “Okay,I'm going to count down from five, put the user's hands to keyboard.”Okay. Now it remembers where the user's keyboard is, right? As the usermove the user's head, it knows where the keyboard is. As the user putthe user's finger on each key, just to configure it, set it up in thebeginning. The user can configure it by just whatever keyboard the userhave. so the user would teach it where the user's keys are. Then itshows the user's keyboard in VR, right, and the user can put the user'shands down, but when the user look down in VR, the user see a model or arepresentation of what the user's real-world keyboard looks like, exceptit is in VR. It's exactly where the user's keys would be, so the usercan look down at the keyboard now. The user have to train it to knowwhere the user's keys are going to be.

Once the user have configured the keyboard once, this configuration issaved in the VR headset local drive and associated to the user's useraccount, so that later, the user are able to use the keyboard with handtracking no matter where the user are. This means a user is able to usethe VR keyboard on their desk, or in space, or anywhere they like.

The user is also able to choose the configuration as they like, so thekeyboard doesn't need to be a traditional keyboard shape. It couldinstead be a bowl like keyboard for each hand, or it could be set on anangle at 45 degrees if the user prefers. The hands could be keptseparate or together, the key distance could be extended; the user isable to configure this however they prefer.

The keyboard can also be configured so that it never occludes what theuser is working on. So if the text is dropping down towards where thekeyboard is, the keyboard is either moved, or the text is adjustedupwards to keep the keyboard clear for the user.

Key Recognition with External Cameras

Use the external cameras on the headset to just read the letters off thekeyboard, with optical character recognition. The cameras can see thekeyboard, they can read letters. It can see exactly where the user'skeyboard is. Then it copies that keyboard into VR at the samecoordinates of where the keyboard is currently. If the user bumped theuser's keyboard forward, the cameras still see where the keyboard is.The VR keyboard is positioned in the same place as the physical keyboardis. This means that the user types on their physical keyboard and getsthe same tactile response as if they were using that keyboard on theircomputer, but the VR keyboard picks up the movements with hand trackingto type in VR.

The VR keyboard is hidden in VR until the user start typing (that is,until hand tracking recognizes the users hands are moving).

The user can also choose where to set their keyboard in VR; once thephysical keyboard has been configured once, then the physical keyboardisn't actually necessary for typing in VR; the user uses hand trackingwithout a physical keyboard which still works (as the head trackingwould register movements against the configured physical keyboard latheuser), but the user just wouldn't get a tactile response without thephysical keyboard. If the user is traveling and the user is in theuser's hotel room and the user has got the user's headset, but nokeyboard; the user draw with the user's finger, using hand tracking, asquare where the user want the user's VR keyboard to appear on theuser's table (using passthrough feature to view the user's physicaltable through the external cameras on the VR headset). If I want to restmy hands on just a blank table, I draw where I want the keys to be. Ican make a book or something, or a calculator, or whatever the user haveor want to use to make/give a tactile response.

VR Configuration, Hand Tracking Configuration with No Physical Keyboard

One embodiment: The user have the user's hands where the user want them,and the VR system just creates the keys where the user want them, usinghand tracking. Then, the user can have a little slider that says how farapart the user want the keys. The user can just slide it with the user'sfingers to set the correct VR keyboard.

Another one is that the VR system just says something like, “Put theuser's hands down on the keys, where they would be.” It sees how farapart the user's fingers are. It says, “Okay, well that's A-S-D-F-J-K-L,semi-colon. Now, tell us where the user want Z-X-C-V-B and M whatever.The user's hands are here.” Now, the user move the user's fingers back alittle bit. Now, occlusion could be a slight issue, but the cameras willstill see the user's knuckles and they see the user's knuckles move upand down a little bit, which will be sufficient to control the handtracking on the keyboard. Actually, the system not only is learningwhere the keyboard is, but it's also learning what the user's hands dowhen the user is trying to get down to a specific key. For example, whenthe user get down to the M key, the user move the user's whole hand alittle bit and move the user's finger up too. The knuckle tracking onOculus is already good enough to pick up this movement, but as a user,the user could train the user's own controls to improve over time; as auser, the user can always overdo it a little bit. If the user had tolearn to overdo a little bit so the cameras could pick it up.

If the external cameras are inconsistent in detecting hand movement inVR for a user, then the keyboard could be adjusted in several ways toimprove the accuracy of the hand tracking control:

-   -   The keyboard is adjusted to have the keys farther apart, which        would increase the distance that a user would have to move their        hand to hit a key, and therefore this would improve the hand        tracking accuracy    -   The keyboard is staggered or staged, like a staircase, so that        the top row of keys (QWERTYUIOP) is slightly up, and the bottom        row of keys (ZXCVBNM) is slightly down, from the middle row        (ASDFGHJKL). This would increase the angle the user would have        to move their hand, and therefore improve accuracy.    -   The keyboard is slanted upward at 45 degrees, which would mean        the user had to elevate their hand slightly to hit each higher        row of keys, again making hand tracking movements more clear and        detectable.    -   The keys in the VR keyboard are made thicker (i.e. they stick        out further in 3D, they are taller), which means that the motion        of depressing a key has to be more exaggerated, again improving        hand tracking accuracy    -   The VR keyboard is positioned one inch ahead of the users hands        at all times. When the user moves their hands, the keyboard also        moves, so that it is always one inch further than the users        hand. This means that, while keeping their palm stable, the user        has to extend their fingers more, making the movement more        exaggerated and thus more able to be detected on hand tracking.        Even one centimeter would make a substantial difference to hand        tracking accuracy.    -   A user may choose to put a black dot on each of their fingers or        finger nails which would improve the hand tracking detection        accuracy.

Touch Pad

Another way to incorporate a VR keyboard which doesn't rely on theexternal cameras to accurately track a user's fingers is a programmed VRapplication of the invention can have a blank mat that has some wiresgoing through it; like a game pad or a touchpad. When the user tap thetouchpad, it senses where the user is tapping, like a sensor in thephysical world, without buttons, just a trackpad with full sensors. Thisis connected to the user's VR headset through Bluetooth or Wi-Fi. Thisstill needs to be configured to set where the keys would be (which theuser would do once, in VR, by using passthrough and setting each key oneat a time.

Device on Fingers

Another option to reduce the need for the external cameras is a devicethat the user wrap around the user's fingers. It's a sensor wrappedaround all 10 fingers (one sensor kit on each hand) and the user trainthe device that way, whereas the user move the user's fingers around,the user train it where exactly the user want the keys. This may also becustomized by each user; if someone consistently got slightly the wrongkey, a programmed VR application of the invention can train the systemto know where an individual would use their hands instead, so that theuser may have a slightly custom keyboard. If someone preferred to movethings even so slightly, or put the enter button up higher or whatever,the user have a customized keyboard.

Using an Additional Camera (Smart Phone):

Another option to improve the accuracy of the external headset camerasfor hand tracking is to add an additional external camera. With the newtechnologies and respective APIs available for the new breed of phones,the possibilities of enhancing the system significantly is very high.Many devices have available SDKs or hardware level APIs to access thesefunctions. This is done with a complimentary smart phone app (forAndroid and iOS) where the user just sets up their phone on their deskwith the camera pointing towards where they want their hands to go. Thisimmediately solved the occlusion issue as the phone camera is set upfrom the opposite direction as the VR headset cameras, thereforecollecting visual data from both sides and improving the hand trackingaccuracy. The VR application in an embodiment of the inventioncommunicates with the user's VR headset through Bluetooth, Wi-Fi and/ora user sign-in to link to the user's VR account.

Customize Virtual Environment for Student

As mentioned above, a first aspect of the present invention is able tocustomize the virtual environment from the student's point of view inorder to make the student more comfortable during learning. In general,features of the environment may be changed by the student but the actualcontent that the instructor presents may not be. Features of the virtualenvironment that by default may be changed by the student include: thebackground of the virtual-reality environment; the appearance of theinstructor, student or any associated avatar; any room in which theinstruction takes place; any objects that the instructor or studentcreated places in the virtual environment.

Regarding the background, this refers to the color, pattern or imagethat the student sees in the background when he or she or she puts onthe head-mounted display. For example, by default, the entire backgroundin whichever direction the student looks (straight ahead, left, right,up, down, etc.) may be black. Or, the student may select a background of“outer space,” giving the impression that he or she or she is standingin outer space with stars all around. Other backgrounds may includeother colors, patterns, images from the real-world such as beingunderwater, etc. By default, any background feature in a virtual-realityenvironment has the characteristic that will appear “behind” anywriting, drawing, room or object that is created in that environment.For example, should an instructor place a virtual blackboard into theenvironment and start writing upon it, that blackboard and the writingwill obscure a corresponding portion of the background.

The appearance of the instructor or student within the virtualenvironment may also be customized by the student. For example, theinstructor may be visible, invisible, transparent, of a different size,or the student may simply choose from a selection of avatars, cartooncharacters, etc., to represent the instructor. Given a particularlifelike representation of the instructor, the student may also changehow old the instructor looks, skin color, facial features, etc.

The student or instructor may also select a virtual room in whichinstruction takes place which is placed “in front of” any background asdescribed above. For example, if the background is outer space, and thestudent selects a wooden floor and a single wall with a virtualblackboard, then the student would see a wooden floor when looking down,a wall with a blackboard when looking forward, but outer space waslooking to the other sides and upwards. Even if the instructor selects aparticular room, the student may change that room by selecting adifferent floor, fewer or greater walls, etc.

Any object capable of being rendered within a virtual-realityenvironment may also be placed into that environment by the instructoror student. For example, a student may place a table, artwork on thewalls, etc.

In contrast to the above features that may be placed or changed astudent within the virtual environment, there is the concept ofeducational content that is placed into the environment by theinstructor or created by the instructor during session. In a simpleexample, the instructor may choose a virtual blackboard to be placedinto the virtual environment, and may then write upon the blackboardusing the stylus 24 or controller 28. Because the invention pertains toa collaborative learning environment, one of the individuals in theenvironment will be the lead individual (typically the instructor) andwill have the capability to designate which content is static content,and which are the dynamic features that a student may change. Forexample, a formula written on the blackboard by the instructor using thecontroller is static content that may not be changed by the student,whereas the background is a feature set by the student that may bechanged by the student. Or, a student may create a table in theenvironment, but a molecule drawn by the instructor placed upon thetable may not be changed by the student.

By default, features such as the background, appearances, etc., may becreated and changed by the student, but any content created by theinstructor may not be changed by the student. In addition, theinstructor may designate certain content able to be changed by thestudent, or, after writing a formula on the board, the instructor mayallow that particular content to be changed by the student (to allow thestudent to rewrite a formula, for example).

A virtual environment may be customized by a student. In thisembodiment, the instructor and student are in different locations, butare able to view the same virtual environment (that each the student maychange) via their respective HMD, making use of either computer locatedwith the instructor, or computer located on a cloud computing platform.

In a first step, the classroom virtual reality environment is displayedto both instructor and student. In a second step, by default, allclassroom features or a subset are tagged as being dynamic features,meaning that each feature may be changed by the student. These featuresare typically tagged when the virtual reality software executes, using adatabase, for instance. Or, the instructor may manually tag each featurewhen he or she or she interacts with the classroom environment using aninput device such as controller 3728. In this example, the classroomfeatures tag is being dynamic are the background and the virtualblackboard.

In step three, by default, any potential instructor input is tag isbeing static, meaning that any such input may not be changed by thestudent. Any instructor content that exists when the virtual realitysoftware begins execution may be tagged as being static (such as a listof homework that the instructor has added at the end of the last sessionor before execution of this session), or, by default, any input createdby the instructor using his or her or her controller will be tagged asstatic.

In step four, the student, using his or her or her controller 3768changes a classroom feature such as the background, or size, location orcolor of the virtual blackboard using his or her or her controller. Boththe student and instructor are able to view this change. In oneembodiment, this change only lasts the duration of the session and if anew session is begun, or if the virtual-reality software is stopped andthen restarted, the classroom feature will appear as if the student hadnot changed. In another embodiment, any change to a dynamic feature madeby a student is saved into a database and that change will be present inany subsequent session or after restarting of the virtual-realitysoftware.

In step five, the instructor draws a molecule within the classroomenvironment using his or her or her controller 3728. Or, the instructormakes use of an interface using the controller to select a particularmolecule that has already been created. Or, the molecule may alreadyexist within the classroom environment (because it had been created andsaved before, or because it is part of the virtual-reality software). Inany case, this molecule is tagged as being static content because it wasdrawn, selected, created etc., by the instructor. This molecule is ableto be viewed by both instructor and students in the classroomenvironment.

In step six, the student is unable to change the molecule because it isstatic content and the student does not have permission. Attempts by thestudent to use any of his or her input devices to change the moleculeare unsuccessful and the molecule will always be visible via the studentHMD 3762. In one embodiment, the student will be able to draw our placeanother object in front of the molecule so that it is obscured. Inanother embodiment, the student is prevented from drawing or placing anyobject in front of the molecule. In step seven, the instructor, usinghis or her or her controller 3728, changes the molecule tag from staticto dynamic. Once changed, the student is then able to change themolecule using his or her or her controller 3768. This step is usefulif, for example, the instructor now desires that the student modify themolecule to produce a different compound. In step eight, the studentchanges the molecule using his or her or her controller 68 to produce adifferent molecule under the direction of the instructor.

First Hardware and Software Embodiment

FIG. 37 illustrates a collaboration system 3710 in which each individualmakes use of a locally available computer. Shown is an instructor 3720wearing a virtual-reality head-mounted display (HMD) 3722, holding anelectronic stylus 3724, wearing a virtual-reality glove 3726 or holdinga controller 3728, in a physical environment such as a room 3730 whichmay also include positional tracking devices 3738, 3739. Student 3760 islocated in a remote location, such as in a different state or country,although it is possible that both instructor and student are in the samebuilding or even within the same room. Student 3760 also wears an HMD3762, holds an electronic stylus 3764, wears a virtual-reality glove3766 or holds a controller 3728, and is in a room 3770 that also mayinclude positional tracking devices 3778, 3779.

Depending upon the technology used, the stylus or glove may be used towrite or draw in the air, their movements being tracked by thepositional tracking device 3728 or 3768. Or, if containing suitablesensors such as a gyroscope and accelerometer, stylus 3724 or 3764 maybe used to write or draw in the air without the need for a trackingdevice 3728 or 68.

Computers 3750 and 3790 are any suitably powerful desktop or laptopcomputers arranged to execute virtual-reality software, render avirtual-reality environment, and display that environment to eachindividual via HMDs 3722 or 3762. For instance, each computer executesthe latest Microsoft operating system, uses a suitably powerful CPU(such as the Intel Core i3-6190), has at least 8 GB of RAM, and has asuitably powerful GPU (such as the Nvidia GeForce GTX 960).

The wired connection 3740 or 3780 between the HMD and the computer uses,for example, an HDMI cable, a USB 2.0 cable and a power cable, in thecase of the Oculus Rift HMD. Internet connections 3742 and 3782 utilizeany suitable Internet protocol and Internet Service Provider so thatcomputers 3750 and 3790 may communicate and pass data back and forth atsuitable speeds.

Second Hardware and Software Embodiment

FIG. 38 illustrates a collaboration system 3810 in which each individualmakes use of a remotely-located computer. Similar to the collaborationsystem of FIG. 37, instructor 3720 and student 3760 each may make use ofa stylus or glove or a controller to write or draw in the air. Each alsowears an HMD 3722 or 3762 capable of wireless communication with a localwireless router 3850 or 3890. Some elements of the embodiment shown inFIG. 37 that are also used in FIG. 38 are not labeled or shown in orderto reduce the complexity of FIG. 2.

Instead of computers 3750 and 3790 rendering a virtual-realityenvironment, cloud-based server computer 3854 is used to execute thevirtual-reality software and to render the virtual-reality environmentfor both HMD 3722 and HMD 3762. Data is transmitted between thiscomputer and the HMDs via connections 3742 and 3782 and the wirelessconnections at each location.

HMD 3722 or 3762 may be any suitable virtual-reality headset that isoff-the-shelf or custom-made, and may include additional cameras,sensors, displays or output. For example, the Oculus Rift, HTC Vive, orSony PlayStation VR may be used. Controllers 28 and 68 may be thewireless controllers “Touch” that are used in conjunction with infraredcamera sensors 3738, 3739, 3778 and 3779, all of which are availablefrom Oculus VR, LLC.

Electronic stylus 3724 or 6374 may be any suitable passive or electronicstylus capable of producing writing that can be transmitted to computer3750, 3790 or 3854, capable of itself transmitting writing or drawingswhen used in the air, or capable of a writing or drawing in the airbeing detected by one of devices 3738, 3739, 3778 or 3779. This stylusmay also be an electronic stylus in that it includes an electroniccomponent in its tip. Another stylus may also include an electroniccomponent such as an infrared light in its tip, the infrared light beingdetected in motion by one of devices 3778, 3739, 3778 or 3779. Thedevice is thus able to detect the writing or drawing in the air bytracking the movement of the infrared light in the tip of the stylus.

Yet another electronic stylus may also include three-dimensionalposition sensors such as an accelerometer, a gyroscope, etc. Thesesensors allow the electronic stylus to be held in the hand and moved inthe air in order to write or draw in two dimensions or in threedimensions. A wireless or a wired connection allows the movements of theelectronic stylus to be transmitted to one of computers 3750, 3790 or3854. Virtual-reality glove 3726 or 3766 may be any suitable glove orhand covering that is suitable for writing upon a reactive surface orthat is able to be tracked by device 3728 or 68.

A variety of virtual-reality software programs may be used to providethe individual virtual-reality environment available in the prior art,and these programs may be modified, or custom programs may be createdusing the information provided herein, in order to produce thecollaboration system of the present invention.

For instance, the virtual-reality software program “Tilt Brush”available from Google, Inc., may execute upon computer 3750, inconjunction with the HMD Oculus Rift, controller Touch, and camerasensors all available from Oculus VR, LLC. These components provide theindividual virtual-reality environment in which instructor 3720 maywrite, draw, paint, etc., in space upon a virtual blackboard.Embodiments of the invention provide the ability for student 3760 toview via his or her or her HMD 3762 the same virtual environment thatthe instructor is creating in real time, and to modify in real time whatthe instructor has written in the virtual environment.

As mentioned above, a second aspect of the present invention allows asingle virtual environment to be displayed in real time to both theinstructor and student, and allows a change made to that environment byone to be viewed in that environment by the other. For example, aninstructor chooses a virtual environment which is simply a blackbackground with a rectangular virtual whiteboard in the middle. Both puton their head mounted displays and both see the same virtualenvironment. When the instructor begins to draw upon the virtualwhiteboard using his or her or her controller, the student immediatelysees that same drawing within his or her or her head mounted display.Similarly, if the student also draws upon the whiteboard, the instructorwould also immediately see that drawing.

A single virtual environment may be shared between instructor andstudent. In this embodiment, the instructor and student are in differentlocations, but are able to view the same virtual environment (that eachmay change) via their respective HMD, making use of either computer 50located with the instructor, or computer 3854 located on a cloudcomputing platform.

In a first step, suitable virtual-reality software executes on theinstructor computer, such as, for example, educational virtual-realitysoftware that simply displays a uniform background and a virtualblackboard, termed the “classroom” environment. In step two, theinstructor is able to view this background and virtual blackboard usingHMD 3722. In step three, student 3760 establishes a connection betweenhis or her or her HMD 3762 and the instructor computer. In step four,the student is able to view the classroom environment that theinstructor is also viewing at the same time.

In step five, the instructor writes a particular formula on the virtualblackboard using, for example, controller 3728. Of course, any othersuitable virtual-reality input device such as stylus 3724 or glove 3726may also be used. The instructor is able to view this formula in theclassroom environment via his or her or her HMD 3722. At the same time,in step six, the student is also able to view this formula in theclassroom environment using his or her or her HMD 3762. In step seven,while viewing the classroom environment with the formula, the studentwrites a solution on the virtual blackboard below the formula usingcontroller 3768, for example. In step eight, at the same time, theinstructor is also able to view this solution in the classroomenvironment using his or her or her HMD 3722.

Various embodiments of the invention have been described. It will,however, be evident that various modifications and changes may be madethereto, and additional embodiments may be implemented, withoutdeparting from the broader scope of the invention as set forth by theclaims. This specification is to be regarded in an illustrative ratherthan a restrictive sense.

What is claimed:
 1. A method for providing an information board in avirtual reality environment comprising: displaying an information boardincluding a first discrete segment viewable in a three-dimensionalvirtual reality environment; receiving a control input adding a seconddiscrete segment to the information board; and displaying the seconddiscrete segment adjacent to the first discrete segment in the virtualreality environment in response to receiving the control input.
 2. Themethod of claim 1 further comprising: receiving first viewableinformation on the first discrete segment and second viewableinformation on the second discrete segment; and storing in a databaseaccessible by the virtual reality environment sequential orderinginformation for the information board that the first viewableinformation is linked to be displayed on the first discrete segment, thesecond viewable information is linked to the be displayed on the seconddiscrete segment, and the second discrete segment was created after thefirst discrete segment.
 3. The method of claim 1, wherein the firstdiscrete segment has a left side and right side from a user'sperspective in the virtual reality environment and the second discretesegment is displayed after and to the right of the first discretesegment following receipt of the control input adding the seconddiscrete segment.
 4. The method of claim 3, further comprising receivingadditional respective control inputs to add respective multiple discretesegments of the information board and displaying respective multiplediscrete segments of the information board in the virtual realityenvironment in a sequential order following first and second discretesegments following receipt of the respective additional control inputs.5. The method of claim 4, further comprising displaying the informationboard in a room with walls in the virtual reality environment andadjusting the size and display of the room when respective multiplediscrete segments are displayed.
 6. The method of claim 5, furthercomprising displaying an animation of one or more discrete segments ofthe information board collapsing into the information board.
 7. Themethod of claim 4, further comprising displaying an animation of one ormore discrete segments of the information board collapsing into theinformation board.
 8. The method of claim 3, further comprisingdisplaying an animation of the second discrete segment of theinformation board collapsing into the information board.
 9. The methodof claim 2, further comprising displaying an animation of the seconddiscrete segment of the information board collapsing into theinformation board.
 10. The method of claim 1, further comprisingdisplaying an animation of the second discrete segment of theinformation board collapsing into the information board.
 11. The methodof claim 1, further comprising displaying the first discrete segment asone of a whiteboard, chalkboard, grid with gridlines, and grid withdotted markers following receipt of control input selecting aninformation board segment type for the first discrete segment from adisplayed graphical user interface.
 12. The method of claim 12, furthercomprising the second discrete segment as one of a whiteboard,chalkboard, grid with gridlines, and grid with dotted markers followingreceipt of control input selecting an information board segment type forthe second discrete segment from a displayed graphical user interface,wherein the first and second discrete segments are different informationboard segment types.
 13. A method for providing an information board ina virtual reality environment comprising: displaying a graphical userinterface for a user to select a first control input to add a discretesegment to an information board in a three-dimensional virtual realityenvironment; displaying a graphical user interface for a user to selecta second control input corresponding to a type of information boardsegment when adding the discrete segment to an information board in athree-dimensional virtual reality environment; and receiving respectivemultiple first control inputs to add respective multiple discretesegments of the information board, receiving respective multiple secondcontrol inputs selecting a type of information board segment for eachadded respective discrete segment, and displaying respective multiplediscrete segments of the information board in the virtual realityenvironment in a sequential order.
 14. The method of claim 13, furthercomprising: storing in a database the sequential order of the display ofthe multiple discrete segments of the information board; and followingreceipt of a third control input requesting retrieval of the informationboard after a user has left and returned to the virtual realityenvironment, displaying a plurality of the multiple discrete segments insequential order in the virtual reality environment.
 15. The method ofclaim 14, further comprising displaying respective viewable informationcontent associated with each respective discrete segment of theplurality of the multiple discrete segments displayed in sequentialorder.
 16. The method of claim 13, further comprising displaying are-sizing of a room in which the information board is displayed in thevirtual reality environment in connection with displaying the respectivemultiple discrete segments of the information board.
 17. A system fordisplaying an information board in a three-dimensional virtual realityenvironment comprising: a graphical user interface displayed in thevirtual environment configured to receive control inputs to add displayof discrete segments of the information board in the virtual realityenvironment in a sequential order; and a virtual room that is viewabletogether with the graphical user interface in the virtual realityenvironment, wherein the virtual room is configured to re-size when anumber of discrete segments of the information board have been added tothe information board that would result in the information boardappearing to extend beyond the walls of the virtual room.
 18. The systemof claim 17, further comprising an information board that includesmultiple discrete segments sequentially ordered from left to right basedon the time of addition to the information board from the perspective ofa user in the virtual room.
 19. The system of claim 18, furthercomprising a database operatively linked to the information board thatstores respective persistent two-dimensional information correspondingto each respective discrete segment of the information board.
 20. Thesystem of claim 19, further comprising a physical controller configuredfor the user to add the persistent two-dimensional information tocorresponding discrete segments of the information board in the virtualreality environment.